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i INTRODUCTION 


The Logistics Management Decision Support System 
(LMDSS) is being introduced to help Naval Air Systems 
Command (NAVAIR) logistics management teams reduce the life 
cycle support costs of aviation systems while protecting 
readiness. Recently LMDSS identified a “bad actor” in the 
F-14 aircraft avionics system. It isolated the Central 
Signal Data Converter (CSDC) and identified a replacement 
with similar functionality at a lower cost over the life of 
the avionics computer. A former F-14 APML stated without 
hesitation that the $42 million cost avoidance could not 
have been found without the LMDSS prototype system in place. 
(NAVAIR 7.0, 1997) Was this an isolated incident? Can we 
expect outcomes like this in the future from the LMDSS? 

Sperating and Suppore (@%S) Costs, that aceotint for 50 
mo oO 6 of the Navy Ss Total Obligation Authority, are 
escalating as aircraft age and compete for the same limited 
resources. (NAVAIR TEAM, 1998) The Navy must improve 
business processes to reduce costs in order to secure the 
resources needed for investments in modernization and 


recapitalization. Cost-reduction initiatives are driving 


program managers to treat O&S costs equal to other 
performance criteria. 

Reducing life cycle support costs depends upon 
effective logistics management and planning that, in turn, 
depends upon tools to support decision-making. The LMDSS is 
a tool to reduce life cycle support costs of aviation 
systems. It is a Naval Aviation Logistics Data and Analysis 
(NALDA) Phase II application that incorporates data from 
existing maintenance, flight, cost and material databases 


into a structured decision making process. 


II. NAVAL AVIATION PROGRAM MANAGEMENT 
AND LOGISTICS DECISION PROCESSES 


A. RESPONDING TO THE POST COLD WAR PERIOD 


The LMDSS is a tool designed to support the Program 
Managers, Air (PMAsS) reduce life cycle support costs by 
supporting decision-making. The Navy must find ways to 
improve business processes to reduce costs in order to 
secure resources necessary to make investments in 
modernization and recapitalization required to carry out 
future missions.! NAVAIR is responding to the post cold war 
PewVccm pmseveral ways (NAVAIR 3.6.1.1, 1998). First, by 
eliminating military-unigue reguirements and procedures that 
aie Up dCQutsit hom COSE€S ENSRCOSt Mo Enairematt Systems and 
associated support is reduced. Second, new policies are 
removing the impediments to getting state-of-the-art 
technology into Navy aircraft and related systems. Advanced 
technology has proven a true force multiplier in the 
development and deployment of weapons systems (Hickock, 
oot Ox eee > dita, Lirms that traditionally produced 
1 See www.acq-ref.navy.mil/pdf/abc.pdf, Navy Acquisition 


Reform for additional readings on modernization and 
recapitalization efforts. 


GOods primarily, if not solely, for the Deparvimencsot 
Defense are encouraged to diversify into commercial markets. 
Fourth, new policies and strategies are targeting the 
reduction of operating and support (0&S) costs. The large 
consumption of resources in this area threatens Navy 
MOoderniZakionmeana FPecapitalizaulon Crheores stile kKoew, ro oy. 
EOxG meso ae 

O&S costs account for between 50 and 60 % of the Navy’s 
Total Obligation Authority (NAVAIR TEAM, 1998). These costs 
are escalating as aircraft age, competing with investment 
reguirements for the same limited resources and thereby 
hindering improvements to infrastructure. Cost-reduction 
initiatives are changing the focus of program managers who 
must now treat O&S costs as they do any other performance 


criteria; systems must be affordable and supportable as well 


as meet operational requirements. 


B. AFFORDABLE READINESS 


Affordable Readiness is the means by which NAVAIR 
intends to significantly reduce 0&S costs while sustaining 
requisite readiness levels. The resulting cost savings and 
cost avoidances can then be directed toward modernization 
and recapitalization. The objective is to meet required 


mission performance and ensure safe operations at the lowest 


ownership cost. Previously the focus of program managers 
was centered on schedule and the projected average unit 
procurement cost with secondary interest on projected 
operations and support objectives. The shift from Design to 
Cost to Cost As an Independent Variable is a philosophical 
shift. The previous approach resulted in maximized 
performance at nearly any cost.* In general, ownership 
costs can be measured in terms of manpower, materials and 
resources. Readiness, availability, Ree ei Emme, Eurn— 
around-time, and other similar metrics measure performance. 
Proposed Affordable Readiness Metrics are listed in Figure 
I 

Affordable Readiness is a business practice with four 
inter-related elements: flexible sustainment, sustained 
mMeantenance planning, rightsourcing, and toval cost of 
ownership. Appendix A describes each element. Analysis of 
naval aviation O&S costs reveals six major drivers that the 
program management team can influence by implementing 
Affordable Readiness: maintenance concept, inventory, 
manpower, technical data, infrastructure, and warranties 
(NAVAIR 3.6.1.1, 1998). Continuous review of in-service 
weapons systems to adjust the maintenance structure based on 


4 See Land, 1997; Kausal, 1996 for a historic perspective on 
CALYy. 
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fleet feedback concurrently optimizes operational 
requirements and provides opportunities to eliminate 
unnecessary costly activities. Reliability Centered 
Maintenance (RCM)?* analysis and Logistics Engineering Change 
Proposals (LECPs) processes help to achieve better inherent 
reliability, target technology insertions, and avoid 
obsolescence. Better inventory management and repair 
process analysis can reduce out of service time for 
aircraft, spares, and support equipment. Smart decisions 
early in the acquisition planning process can reduce 
material and manpower requirements to support an aircraft 
system throughout its total life cycle. Partnerships with 
industry, consolidating capabilities, the use of digitized 
data, single process initiatives, reinvention initiatives, 
reliability warranties, and integrated diagnostics are some 
of the many cost-effective’initiatives. Program managers 
must make intelligent trade-offs between performance and 
ibbre "evyele COSTS, wine decision SUppoOre = syStEems MUSt SUDPOFE 


the PMA developing and analyzing alternatives. 


3 See www.nalda.navy.mil, NAVAIR Logistics, Affordable 
Readiness Link. 


Ce. PROGRAM MANAGER ROLE IN AFFORDABLE READINESS 


Figure 2 depicts organization responsibilities as they 
pertain to Affordable Readiness. Affordable Readiness is a 
management approach being implemented within the existing 
organization structure.4 The PMAs are responsible for 
developing plans to implement and execute Affordable 
Readiness; identifying specific reduction targets and 
metrics for tracking progress; setting priorities and making 
investment trade-offs; directing the actions of supporting 
teams like Fleet Support Teams and Integrated Product Teams; 
interfacing with support environments such as policy, 
Buecess, facrlity and @mrrastructure Ozgqanizalions to 
develop optimal policies and processes; reporting actions 
taken and results; and obtaining fleet feedback on how the 
system is performing. Assistant Program Managers, Logistics 
(APMLs) advise PMAs on logistics matters. The program 
management office is where the rubber meets the road in life 
cycle management and total cost of ownership analyses. It 
is here that the manager who has both authority and 
responsibility is held accountable for effective and 


efficient allocation of resources. The PMA must balance 


4 See www.navair.navy.mil, NAVAIR for additional readings on 
the NAVAIR TEAM history and organization. 
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short and long=term objectives and vie for critically 
limited resources. Additionally, he must do so while 
working within a labyrinth tangled with political and 


bureaucratic processes and pressures. 
1D; INTEGRATED PRODUCT AND PROCESS DEVELOPMENT 


PMAS manage within the context of Integrated Product 
and Process Development (IPPD) which is a management process 
that integrates all activities from product concept through 
production/field support. It uses a multi-functional team 
to optimize the product and its manufacturing and 
Sustainment simultaneously to meet cost and performance 
objectives. IPPD evolved from concurrent engineering and 
the philosophies of quality management. It is a system 
engineering process integrated with sound business practices 
and common sense decision making. 

Integrated Product Teams (IPTs) are the means through 
which IPPD is implemented. Appendix B describes the three 
types of IPTs. These cross-functional teams are formed to 
deliver a product with common performance objectives using a 
common approach for which they hold themselves mutually 
accountable. Members of an IPT represent the technical, 
Manufacturing, business, and Support Orgamizatwens that are 


critical to developing, procuring, and supporting the 


10 


product. Team members work together to achieve the team’s 


objectives. 
E. LOGISTICIANS AND INTEGRATED PRODUCT TEAMS 


As functional area experts logisticians have special 
responsibilities because they bring special knowledge and a 
special point of view to the effort. The degree to which 
these experts are willing to share their knowledge and point 
of view will determine their value to the team. In addition 
to providing an expert opinion, experts play an important 
training role on the team. By sharing their expertise, they 
educate fellow team members to the not-so-obvious 
implications of programmatic decisions and actions. Team 
member involvement includes active participation, effective 
communication, challenging requirements that do not make 
sense, and paying attention to detail. For the logistician, 
dedication to these principles can make the difference 
between fielding a costly, ineffective, inefficient system 
See o vec oOne 7 OOEIMa) “Semietons are often a result soe 
well-researched opinions, constructive conflict resolution, 
and tenacity. 

Because evielmicwmexCepEonoumost OlWEhe COST MOEA 
program 1S in the cost of ownership, i.e. the support of the 


Systeme tntrougioure duswooperational life, “the logisti¢ianm can 


pel 


make major contributions t@y the acquisitionger aeaecse— 
effective system. While dealing with short-term problems, 
the logistician must also think about problems that will 
arise in the future. For example, increased environmental 
awareness and legislation has increased the difficulty and 
cost of demilitarization and disposal of systems. An 
unreliable system with poor maintenance support will drive 
the need to procure costly spare components that may or may 
not be available. Identification of such problems early in 
the concept exploration phase of a program can help avoid 
serious consequences later in the program’s life cycle. 

ine logistician’ Simelesen, angueidepengseenernecmayoceor Les 
it 1Ss.9On a higher level IPT nOtedirectly focused on 
Support issues the logistician should be concerned with 
identimiying and highlighting the long-term logistics 
implications of various program issues. He may then form a 
Supportability IPT to focus on mitigating the effects of 
those Pssues on the supportability of the system. At thie 
program level the logistician should be more concerned with 
influencing the design of the system and the design of the 
Support structure. An important responsibility of the 
logistician on an IPT is to help the team create 
Supportability performance requirements that are 


quantifiable and testable so that the decision-makers can 
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dain insmigineinioms NeMOpetabtonalesmapabilitycofwthe product 
and the logistic planners can plan for the support of the 
item. 

The Fleet Support Team (FST) is an example of an IPT 
directed to interface with the fleet, identify problems, and 
develop solutions. The focus of these teams is on all 
aspects of life cycle support for a system from when it is 
fielded until it is disposed. Within the context of 
Affordable Readiness, the FST is the center for identifying 
initiatives, performing analyses, and developing action 
plans. The resulting action plans can include investments 
(Engineering Change Proposals), process improvements 
(consolidated maintenance activities), or innovative support 
solutions (commercial or joint service support equipment). 
IPTs, including FSTs, are not standardized. Each PMA 
determines how he will manage his program. Some teams are 
highly specialized while others cross diverse functional 
barriers. Some FSTs are organized within the program office 
while others are organized within the aircraft controlling 
custodian or depot maintenance engineering support 
organizations. Accordingly, the APML has different 
relationships with teams, both within and beyond the program 
office, as determined by individual program office 


Organization. Additionally, program offices use a number of 


ILs 


different sources for logistics support. Some program 
managers rely heavily on Navy personnel assigned within the 
program office, others rely on government employees from a 
logistics competency group in NAVAIR,°? while some contract 
out to commercial sources for logistics analyses and 


recommendations. 


1 CHANGE FROM CHECKLISTS TO INTEGRATED 
FLEXIBILITY 


Because acquisition program management is tailored to 
meet individual program needs, the challenge of supporting 
information systems is to gather and present data in an 
equally flexible manner. The data must reflect performance 
and supportability metrics in a meaningful, effective, and 


efficient way. 


> Within NAVAIR’s matrix organization in which team members 
report to both a functional leader and a program leader, 
competency leaders are responsible for providing skilled, 
knowledgeable members for IPTs and for managing the 
processes by which these personnel support the teams. The 
Logistics Competency Center’s mission is to provide the 
people, skills, knowledge, processes, facilities, and 
equipment required to manage and perform the planning, 
development, acquisition, integration, and delivery of all 
integrated logistic support elements necessary to affordably 
design, support and maintain weapons systems throughout the 
program’s life-cycle. Supporting missions include technical 
publication development, logistics elements integration, 
affordability studies, and engineering technical services to 
name but a few. 
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Traditional Integrated Logistics Support (ILS) used a 
step-by-step analytical process that defined all the 
logistics and maintenance tasks, resources, and requirements 
necessary to establish and sustain an effective support 
program over the life cycle of a program. The logistics 
community depended on ILS products generated from the 
application of the Logistics Support Analysis (LSA) process 
acuoumpulated in MIL-STD-=-138ce—ure Now as DoD 5000.2-K 
stipulates, supportability factors are to be integrated 
elements of program performance specifications, but support 
requirements are not to be stated as distinct logistics 
elements. Instead they are to be stated as performance 
requirements that relate to a system’s operational 
effectiveness, supportability, and life cycle cost 
reduction. The challenge to the PMA is to develop a 
De sOrMance —lacedustarement (Of Work taatuimeludes 
supportability metrics in addition to the usual 
operationally oriented performance goals. Programs must be 
able to be evaluated on specific performance metrics 
describing the relation of cost-of-ownership with other 
parameters such as mission performance, safety and 
availability/readiness. Appendix C describes four factors 
that must be considered in establishing supportability 


requirements. 


ls 


MIL=HDBK-SO2 ofters guidance om, aequi sit tonmmeemories 
as an integral part of the systems engineering process. The 
information is applicable, in part or in whole, to all types 
of material and automated information systems and all 
acquisition strategies but it is not a “cookbook” approach 
to acquisition logistics. It is intended to accommodate the 
vast, widely varying, array of potential material 
acquisitions and provides general guidance to logisticians 
on how to perform certain aspects of nee jobs: 

NAVAIR has a companion “Contracting for Supportability 
Guide.” This guide identifies five steps (Appendix D) to be 
used to establish a supportability strategy for acquisition 
programs for new systems, major and minor modifications or 


upgrades, and commercial and non-developmental items. 
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III. LOGISTICS MANAGEMENT DECISION 
SUPPORT SYSTEM 


A. HISTORY AND ORIGINAL REQUIREMENTS 


The LMDSS requirement evolved from NAVAIR and Aviation 
Supply Office (ASO) initiatives to assess the logistics 
health of programs using standard metrics of readiness, 
supportability, and cost. In 1991, the effort was expanded 
into a requirement to develop a decision support system 
which would be the primary tool for APML/Weapon System 
Managers (WSMs) to achieve a “continuous, measurable 
reduction in life-cycle costs while maintaining operational 
BeaGdiINess sand SUStainability. _(LMDSS Req. Doc. 1993) 

The driving requirement behind LMDSS was the need for a 
tool to facilitate measurement to plan ene erGdentiricatlon sof 
targets for cost reduction. This would be accomplished 
through use of a software package operating on several key 
databases organized in a relational environment. The key to 
Successful operation of the DSS rested with the integration 
of diverse databases into a central repository. This 


integration would result in an immediate, precise response 


iy 


to queries, regardless of the type data requested or its 
Ole ae aL a 

Access to logistics data ina relational environment 
allowed a level of analysis that was impossible in personal 
computer based systems. Where analytical requirements 
involved databases outside the repository, access would be 
made automatically as a standard function of LMDSS. The 
repository would encompass data from all three maintenance 
levels, the supply community and selected sources of 
specialized information. To speed access, the system was to 
be established using both local and wide area 
interconnection techniques. The system was to be designed 
tO accommodatem@Managers and analysts who were nor 
necessarily Automated Data Processing (ADP) experts. 

The NAVAIR development team determined that to provide 
maximum utility, LMDSS needed to provide both a structured, 
modular approach and an unlimited ad hoc query capability. 
A requirement for an extensive repertoire of Statistical 
PROCESSeCONUROl Jana LbaGlelonaleenareimG, availa lemon 
semi-automated basis was established to compliment both of 
these capabilities. 

The plan was for the data Pepositony co be Pocatca ae 
ASO to provide all Naval Aviation maintenance personnel, 


engineering personnel, supply personnel and procurement 
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personnel a “common and integrated sheet of music.” (LMDSS 
Req. Doc., 1993). Provisions would be made to integrate 
Streamline Alternative Logistics Transmission System (SALTS) 
data with traditional Aviation Maintenance and Material 
Management (AV3M) reporting when available. The objective 
was to incorporate all necessary AV3M and Uniform Inventory 
Control” Poime (UICP) data, so that it could be manipulated 
directly by the logistician, to support detailed causative 
research. The NAVAIR development team also desired summary 
data, where refined and available. However, the focus was 
to remain on ability to support detailed research in a 
relational database management system environment. 

The LMDSS software was to be divided logically into 
five modules of 1) Candidate Selection; 2) Problem 
Isolation; 3) Ad hoc Queries and Special Summaries; 4) 
Evaluation and Selection of Alternatives and 5) 
Implementation and Status Tracking. 

The original requirements for LMDSS assumed a IBM RISC 
System/6000 Model 970 machine located at ASO hosting a 
massive - hundreds of gigabytes - database and regional IBM 
RISC System/6000 machines located at each Naval Aviation 
Depot (NADEP) and selected Naval Air Warfare Center (NAWC) 
activities. The regional machines would not house the 


extensive data held in the NALDA ASO IBM RISC System/6000 
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machine. The regional machines would tap the ASO database 
resource via remote procedure calls. Connectivity between 
the RISC machines would be provided by either direct 
connection of each RISC system to the NAVNET Internet or 
directly 0 the DDN MILNET&) Lt.wouldvenployveasclitent, seuves. 
distributed architecture linking these computers via TCP/IP 
in a WAN and LAN environment. ASO LMDSS customers would 
directly interact and conduct X Windows client/server 
sessions with the ASO RISC machine via LAN and TCP/IP. All 
other LMDSS customers would directly interact and conduct xX 
Windows client/server sessions with their respective 
regional RISC system via LAN and TCP/IP. It was planned as 
an information system requiring direct LAN connectivity. 
The original system was not planned to support PCs or 
workstations in stand-alone mode, nor support connectivity 
via modem. 

All programming was to be done in Ada. The operating 
system for the RISC machines was to be IBM AIX and Oracle 
was to be the database. Rapid prototyping was used for the 
software development methodology. The prototype system is 
running on a RISC machine located at ASO. 

By 1995, it was determined that Ada was unsuitable as a 


host language. HTML and PERL were used to request and 
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display reports. xX Windows and C were being used for 
programming complex displays. 

In 1996, the scope of the LMDSS underwent substantial 
change. The LMDSS database was expanded to essentially 
become NALDA Phase II (NAVAIR 3.6.2, 1998). The LMDSS did 
not start out as the heart of NALDA II but at that point, 
Siw louver nemoyotem became. Appendix HE provides ma 
detailed discussion of the evolution of NALDA and the 
composition of NALDA Phase II. 

in i9oy cue to Gentractor diftficulGucs and “equupment 
problems, the development team decided to convert reports to 
operate with commercial browsers, abandon X Windows and use 
Active X controls for complex reports, and move to Internet 
access. Additionally, they decided to transition from the 
IBM RISC/6000 machines to multinode IBM Scaleable 
POWERparallel2 (SP2) for the production platform. The SP2s 
will all be located at NAS Patuxent River Maryland, rather 
than the distributed architecture originally planned. 
Expectations for what the system would ultimately deliver 
were scaled back. Rather than being everything for 
everyone, core capabilities were identified, and “nice to 
haves” were eliminated. Those capabilities eliminated 
included graphic capabilities, the Return on Investment 


module, and ad hoc query access against the detail data. 


ae 


The ad hoc query capability would still be possible using 
the IQ Tool - an OLAP software product. Because of the 
detailed knowledge of the database and SQL skills required 
to make effective use of this software, the tool would be 
available only to certain users. These users would 


primarily be analysts at the major claimants or the NADEPs. 


B. THE CURRENT STATE OF THE LMDSS 


The data load into the LMDSS/NALDA II database on the 
SP2 equipment is now in its final stages. When this load is 
complete the application will be pointed towehWe new tables 
and turned over to the Fleet as the replacement for NALDA 
Phase I. Automated database loading software is operational 
and AV3M SALTS submissions are now being loaded directly 
into the LMDSS/NALDA II database. Naval Sea logistics 
Command (NSLC) is now out of the AV3M business (NAVAIR 


3. OS oF) 


C. THE LMDSS FEATURES AND CAPABILITIES 


The LMDSS is organized into seven functional areas. 
The following descriptions of these areas and subareas have 


been derived from the LMDSS homepage: 


Za 
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Management Analysis 


This module consists of various tools for displaying 


high level summary data for end items, claimants or 


organizations. These tools identify system degraders and 


produce reports ranked by parameter. The reports include: 
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End-Item Matrix. This produces a matrix that 
Summarizes reliability, supportability and cost data 
to the end-item level. 

Claimant Matrix. This orontieee a matrix that 
summarizes reliability, supportability and cost data 
to the claimant level. 

End-Item/Claiment Matrix. This produces a matrix 
that summarizes reliability, supportability and cost 
data to the end-item and claimant level. 
Organization Matrix. This produces a matrix that 
Summarizes reliability, supportability and cost data 
to the organization level. 

Beyond Capability Maintenance (BCM) Report. This 
produces a report that summarizes BCM data to the 
organization level. 


Candidate Identification 


This module consists of various tools for displaying 


reliability, supportability, and cost summary parameters for 


Ze 


selected aviation equipment. These tools identify system 


degraders and produce reports ranked by parameter. The 


primary purpose of this module is to allow managers to 


identify opportunities for life cycle cost and readiness 


improvement. These tools include: 


Reliiabittey/ Supportability /Costaimas ©) Maras 

Pit cme nt CaS ac (Clove Ob ENheo moa smemmic eh teos: 

Component by Reported NIIN, NIIN Head of Family, and 

Wemeeunit Code (Wve, 

Common Equipment Matrix. This identifies potential 

problems with cross-platform components. 

Additionally, there are four methods for collection 

of equipment/systems/components that have multiple 

alrcratt applicabality: 

® Local Routing Code (LRC). This uses the ASO local 
POM bane code. This selection will only feollect 
data for NIIN or NIIN Heads of family that match 
the specified LRC. 

" Type Model. This collection method is based on 
the minimum number of Type Models in which a NIIN 
must occur to be considered common equipment. 

» Type Model Series (TMS). This method is based on 
the minimum number of TMSs in which a NIIN must 


occur to be considered common equipment. 
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* NZIN. This selection displays the selected NIIN, 
its nomenclature, the Type Equipment Code 
(TEC) /TMS it can be found on, and the number of 
the selected items in the TEC/TMS. 

e Emergent Problems Matrix. This identifies items 
that show recent changes in reliability, cost, 
maintenance and supply. 

e Bureau Number Matrix. Displays support costs, 
maintenance and supply statistics, and reliability 
figures broken out by individual bureau number for a 
Trse 

Sc Trend Analysis 

This module consists of tools used to analyze system 

degraders to determine the basic problem(s) and examine the 
underlying cause(s). The Historical Trend Analysis tools 
display reports of statistics over time. This is tabular 
information summarized by month covering End Item or the 
component levels. 

e Aircraft Utilization History. Presents a parameter 
value entry and selection form to prepare for the 
display of flight and utilization data that aids in 


historical trend analysis of aircraft. 
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4. 


Work Unit Code History. Presents a parameter value 
entry and selection form that includes selection of 
WUCs, to prepare for the display of maintenance data 
that aids in historical trend analysis of WUCs. 
Intermediate Maintenance Activity. Presents a 
parameter value entry and selection form which 
includes input of NIIN and Date Range to prepare for 
the display of intermediate level maintenance that 
aids in trend analysis. 


Cost Analysis 


This module contains tools used to analyze end-item and 


component cost data. 


Annual Operations and Support Costs (AIR 4.2). 
Displays platform level cost information based on 
the OPNAV Code N889 sponsored Navy Flying Hour 
Program. The report can be generated for a specific 
TMS squadron manned at 90% or 100%. 

Budges Analysis (OP-2Z0 Repore) =) Displays the eudgee 
Analysis Report selection parameters. The operator 
selects the fiscal year, TEC and funding command to 


produce the desired report. 
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Labor Cost History. Displays Labor Cost History 
data in tabular format. Parameters of year and 
maintenance level may be selected. 

Inflation Factors. Displays fiscal year, inflation 
rate, raw index, weighted index and budget year 
multiplier for a variety of operator selectable 
appropriations categories. 


Item Value Cost Reports. Allows users to select 
perveecn fie heen Valtemeo Depot ReEpalr Cost report 
or the Item Value to Labor Cost report. In the Item 
Vee Hom PC OOM Nepali Cosme report Ene lise m sean 
compare the replacement value of items in the 
database to the cost of level 3 maintenance for a 
selected time period. The result 1S shown as a 
percentage showing the level 3 maintenance cost of 
in service units compared to the cost to replace 
those units. The Item Value to Labor Cost reports 
are Similar in that they compare the replacement 
cost of items to the labor cost at maintenance 
levels 1 and 2 for those items. The results show 
the annual cost of ML1 and ML2 labor as a percentage 


of item replacement cost. 


Za} 


De 


Detailed Analysis 


This module contains tools used to analyze items 


identified in the Candidate Identification Function. 


6. 


Detailed Component Report (DCR). This is a 
comprehensive report encompassing data from all 
three levels of maintenance (AV3M) and supply 
(Weapon System File/UICP). It is designed to fault 
isolate candidates from the candidate We Chieint Cat cm 
to the root hardware cause(s) of R/S/C degradation. 
This is accomplished through drilling down through 
progressively more specific forms to lower levels of 
detail. 

Supply Synopsis. This section of the Detailed 
Component Report provides greatly expanded 
information covering supply and depot repair 
parameters. 

Source Doemment Repore (VMUS/MAF). This Sseerion 
provides detailed report information of VIDS/MAFs. 
It is possible to drill down to and view a specific 
VIDS/MAF. 


Supply Analysis 


This module consists of specialized summary and 


forecasting reports intended for use by supply personnel. 
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This utility provides a means through which both readiness 
and cost factors are examined concurrently. 

e Wholesale System Demand. This form is used to aid 
in forecasting demand for specific NIINs. It will 
link to a NIIN Analysis screen that will provide the 
user with a breakdown of more NIIN specific 
information and links to the DCR and Tools Cross- 
Reference report. 

e Wholesale System Investment. This report is sorted 
by NIIN and a break out of repair costs is 
displayed. 

e End-Item Material Issue Trends. This report can be 
displayed by TEC or TMS. 

e Average Customer Wait Time Reports. These reports 
can be produced for specific TECs broken out by 
maintenance level, response time crossed with COG or 
the highest wait days across all NIINs for that TEC. 

e Wait Time Maintenance Impact. Under Development 

e Average Days to Receipt. Under Development 

e Backorder History Report. Allows for the display of 
data for TEC/TMS sorted by NAVICP Material Type and 
Data Elements. The report may also include DLA 


Supply Center and Data Elements. 
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e Backorder Ranked Report. This report allows for 
identification of the NIINsS with the largest number 
of backorders against them. 

e Planned vs Actual Opportunity Cost. This form 
allows the user to enter source of reliability data 


to report on from NAVICP, LSAR, or Manually Entered. 


e Mean Flight Hours Between Failures Report. This 
form and resulting report can be used for “what if” 
analysis. Planned data can be entered and then 
compared to actual data. 

e NAVICP NSN Snapshot. This report may be generated 
for a specific NIIN. Either a summary report or a 
report specific to backorders, stock status, 
alternate NIIN, etc. may be produced. 

e Repair Cycle Report. Gathers and displays data on 
repair cycle and BCM rates. 

a Engine Analysis 

This module consists of tools that allow the analyst to 

view projected actual costs and hours for different engines. 

e Depot Engine Repair Cost. This provides the analyst 
with historical information on the cost, funding 
source, and activity for each engine. This report 


1S not accessible wee conmeractonmc . 
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Engine Overview. The overview report expands the 
information available to include engine inventory, a 
cost breakout and the capability to select data 
covering all TMSs for specific TMSs, all sites, PAC 
Sites or LANT sites. 

Engine Removal Analysis. Tracks average engine time 
Since last repair when removed. The repair site and 
Powe owen eieg esa teGtOuLed tO thal Spectre 
Site are also listed. 

Engine Removal Trend. Charts the engine removals to 
TMS based on 100-hour service intervals since last 
repair. Multiple series may be compared on the same 
Ghare, melving thsightamneO PaGrtOrs Such as  initant 
morlality and “high time.” 

Top Reasons for Removal. Displays the top reasons 
for engine removals -tto TMS. 

Pitght Hours simee last Repairs Thais 42s a companion 
report to the Engine Removal Trend Report. In 
addition to charting removals by TMS, it also shows 
the reason for each removal along with flight hours 


in 100-hour increments. 


Flight Hours Since Repair at Removal. Displays 


engine removals and maintenance man-hours. This 


ou 


report differentiates between scheduled and 
unscheduled maintenance, and displays average hours. 
e Engine Demand Forecasting. Based on the premise 
that evolving reliability and maintainability data, 
both historically derived and imposed, must be 
considered in conjunction with historical demand for 
successful engine demand forecasting. This option 
derives from historical files and extrapolates past 
performance to future needs through application of 
relative flying hours. 
8. Reference Information 
This module consists of tools that provide general 
alTrerareEeInitoOrmakion, —ehanminoncrestaristiecseeassistance, 
reference information/reports, and information about the 
application/database. 
9. Application Management Tools 
Cemeadins variteus utilities) Enat provide current 
database status, user identification, server status, and the 
software Change Request form. 
10. Feature Synopsis 
This provides a brief description of the modules and 


links to the sub-elements. 
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The LMDSS has an on-line Help capability. Each input 
form has an accompanying Help function that explains the 
input fields. Additional links within some of the Help 
screens provide information on algorithms used to derive the 
reports. Other links provide definitions of acronyms and 


computer jargon. 
D. QUALITY ASSURANCE (QA) PLAN 


Prior to the transition from the NALDA I to NALDA II 
database, a systematic series of tests to ensure proper 
functionality, accuracy and a smooth transition is planned. 
The two primary targets of this testing are to assure 
accurate data retrieval throughout the application and 
database integrity. 

Sections of the application and certain derived data in 
the Oracle tables where problems have been previously 
identified are being re-coded. Concurrent with this effort, 
the QA team, composed of analysts conversant with LMDSS and 
NALDA I will go through each section of the LMDSS 
application in a critical review of form, format, and 
Becuracy OF COntent. 

The prototype application software currently in use is 


identical to that which will operate against the SP2s. This 
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means that quality assessment can begin immediately. There 
are specific challenges that must be addressed: 

ie Data Outputs are not Identical 

Although the database which LMDSS now reaches is 
based on a data pull from NALDA I, the data outputs are not 
identical. This is because in LMDSS, the data pre- 
processing has been improved to provide greater accuracy. 
Examples of the differences include use of aircraft versions 
in additw@em to ITMS“amd revised item count logie. this makes 
comparative analysis difficult, but not impossible. 

Ze Software Change Requests System 

The problem is that the reporting system, which was 
built into the client-server version of LMDSS, became 
inactive during the transition to the Internet based 
version. It has been two years since a Software Change 
Request (SCR) has been classified, scheduled or answered. 
This has resulted in a lack of systematic documentation of 
problems and resolutions during the transition (Jones, 
oC 

The SCR system is again working. All SCRs and their 
status can be viewed under the Application Management Tools 
Module of the LMDSS application. An examination of the 
current list of SCRs indicates that there has been 


significant progress made on the application validation. 
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E. ANTICIPATED ADVANTAGES 


The development team anticipates that the LMDSS will 
provide an important analysis function to assist logistics 
managers in establishing requirements for new acquisitions 
as well as troubleshooting existing weapon systems. Because 
of the Integrated Data Environment (IDE), the LMDSS will 
allow all potential user levels to substantially reduce the 
amount of time required to identify and analyze problems in 
Feoglstles SUDpOLt . 

The LMDSS will provide an ability to analyze data 
concerning common equipment - those items that are used on 
more than one weapons platform. The present approach 
essentially looks at present conditions and backordering 
philosophies that encourage a tunnel vision approach due to 
Serr ecCuley sim identifying common components that can be used 
across platforms/weapon systems in correlating 
maintenance/repair and ordering of specific common 
equipment. 

With the implementation of the LMDSS, daily feeds of 
maintenance and flight data will be received through SALTS 
So miiMialswalreerlyObO ENG production database. The current 
Svyorven USCS ea smoOnecalysmpaare cycle based upon the NSLC 


processing schedule that establishes data as 45 days old as 
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the best case. Monthly reports will be available at least 
30 days earlier under the new system. Data quality will 
also be improved with the strengthening of validation 
specifications at the source and use of “most probable 
logic” within the data summarization funcevon oF the 


database. (NAVAIR 7.0, 1997) 
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IV. LITERATURE REVIEW 


A. THE DECISION SUPPORT SYSTEM AREAS OF RESEARCH 


Much of this DSS research can be classified into one of 

five areas: 

1) what distinguishes the DSS from other computer 
information technologies; 

2) whether the DSS actually improves decision quality 
or performance; 

3) idemerivyinege@specmevemdesitign characteristics and the 
impact they have on the DSS development; 

4) the role of the decision-maker and how differences 
between individuals and organizations can influence 
the effectiveness of the DSS; and 

5) 2DSSeevaluation acu: 


ow The DSS Versus Other Computer Information 
Technologies 


The first area of research has addressed what 
distinguishes the DSS from other computer information 
technologies. Currently, there is a general consensus that 
a DSS is composed of the following three interrelated 


components: data management, model management, and dialogue 


au 


management components (Alter, 1977, Sprague, eis, een: 


eS ee 


Each component provides specific capabilities to a 


decision-maker and improves the effectiveness with which he 


or she works. 


The data management component should include: 


1) 


Za 


the capture and extraction of data into a database; 
the storage, retrieval, and control of data by a 
database management system; 

the ability to interact with data from internal and 
external sources; 

the ability to perform ad hoc queries; and 

the flexibility to allow rapid additions and 
changes in response to unanticipated user requests. 


(Al@er, 977; Spreaeue, 1980; Keen, 1931) 


The model management component of DSS provides a user 


with a set of capabilities that differentiate it from other 


traditional computer systems. These capabilities include: 


1) 


the use of multiple models to support diverse 
problems; 

CHRENSUPDSEL Of Semi-structured andwunstructuneca 
problems; 

the ability to bud ldemedelssquickivys andecasi i 
the ability to track models through a model 


directory; 
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5) the ability to integrate models with appropriate 

links through the database; and 

6) the creation, retrieval and storage of models 

handled by a model base with management functions 
analogous to database management. (Alter, 1977; 
Sprague, 1980; Keen, 1981) 

The dialogue management component provides the mode of 
interaction between the user and the DSS. Research results 
suggest that a well-developed interface should include: 

1) the support of multiple dialog styles; 

2) the capture, storage, and analysis of dialogue 

through a dialog management system; 

3) the ability to interact with the model and data 

components of the DSS; and 

4) the support of multiple methods of presenting 

OUEDUEMEO provede for a varleby of Lormats and 
media, and the flexibility for different users’ 
knowledge base. (Alter, 1977; Sprague, 1980; Keen, 
Gy) 

ae DSS Benefits 

The second group of studies has primarily focused on 
benefits that the DSS provide. The primary justification 
for the development of a DSS is that it will be a value to 


the decision maker (Hogue and Watson, 1983). Some studies 
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in this area support the premise that the DSS improves 
decision quality or effectiveness (Sprague and Watson, 1986; 
Hogue and Watson, 1985; Sprague and Carlson, 1982; Keen and 
scott Morton, 1978). In other studies there was no effect, 
and in still others decision quality worsened when a DSS was 
employed (Benbasat and Nault, 1990; Sharda, et al., 1988). A 
DSS provides a coherent strategy for going beyond the 
traditional use of computers in structured situations where 
measures of effectiveness and efficiency are nearly 
equivalent.® In the semi-structured and unstructured 
Situations in which the DSS is used, effectiveness has been 
the primary focus. Other research suggests that it is 
efficiency that is actually improved (Todd and Benbasat, 
7 

The DSS is designed to be an interactive system used by 
managers with little experience in computers and analytical 
methods to help improve the effectiveness and productivity 
by supporting, rather than replacing, judgment (Fedorowicz 
and Manheim, 1986). A DSS is, in effect, an assistant to 
whom the manager delegates activities involving retrieval, 
6 Efficiency is performing a given task as well as possible 
in relation to some predefined performance criterion. It is 
a measure of resources utilized against results derived. 
Effectiveness involves identifying what should be done and 


ensuring that the chosen criterion is relevant. It is the 
degree to which a goal is achieved. 
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computation, reporting, and development of alternatives. A 
manager evaluates the results and chooses the next step. 
The benefits the DSS provides are often not quantitative. 
Benefits include the ability to examine more alternatives, 
stimulate new ideas, improve confidence in the decisions, 
reduce the probability of error, improve communication of 
analysis, and speed up decision making. (Keen and Morton, 
1978; Keen, 1986; Hogue and Watson, 1985; Hogue and Watson, 
P63) 

Se Design Characteristics 

ie Sema toamG tl hese ace mac mDeel Cilteeled 
towards identifying specific design characteristics and the 
impact they have on the DSS development. Topics 
investigated have included the impact of different 
presentation formats, the use of color, the influence of 
different graphics capabilities, and the influence of 
different user interfaces. Analysis has generally provided 
mixed results as to the impact of these factors on decision- 
making effectiveness (Bennett, 1983; Pearson and Shim, 
fee). This is not to infer that a DSS that a more easily 
accessible, as with a web browser, does not provide 
measurable benefits. Intuitively, the more accessible the 
design interface, the more positive the impact will be on 


decision-making effectiveness. Because internet browser 
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technology has only been introduced since the mid 1990’s, 
there is a lack of research into the relative effectiveness 
of this interface versus others. 


4. The Role of Individual Decision-Makers and 
Organizations 


The fourth group of studies has addressed the role of 
the decision-maker and how differences between individuals 
and organizations can influence the effectiveness of DSS. 
Research includes theoretical studies of organizational 
decision-making. Specific characteristics investigated 
include cognitive biases and processes, novice/expert 
effects, models of decision-making, and user-situational 
variables (Mittman and Moore, 1984; Mann et al., 1986; Keen 
and Morton, 1978; Alavi and Joachimsthaler, 1992). User- 
Situational variables include user involvement, training and 
experience. The emphasis in this area is on describing the 
methodologies and differences of decision-making so that 
computer technologies can be effectively prescribed and 
applied to improve how decisions are made. Research has 
also found that MIS success is dependent upon the extent to 
which it fits the organizational environment (Raymond, 1990; 
Bin-Dor and Segev, 1978; Ein-Dor and Segev, 1982; Schultz 


and Slevin,. 137si2 
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a The DSS Evaluation Methods 

The last major area of research is that of measuring 
the implementation success of the DSS. Implementation 
success refers simply to realizing the intended benefits of 
the system. Currently, no single approach to the definition 
of DSS implementation success exists in the literature. A 
variety of different variables have been proposed and tested 
as indicators of success. These include such things as 
system use, decision-making time, decision-making 
performance, user satisfaction with the system, user 
confidence in the decisions, and user attitudes towards the 
DSS. (Alavi and Joachimsthaler, 1992; Money et al., 1988; 
Raymond, 1990; Lee et al., 1995; Swink, 1995; Goodhue, 1995; 
Gatian, 1994) 

The evaluation of DSS is a research direction mentioned 
by almost every author in this field, but measuring the 
effectiveness of these systems is a difficult task (Udo and 
Davis, 1992). Again, the literature indicates little or no 
consensus as to a model or methodology for evaluating 
success. Those proposed have progressed from the 
traditional cost-benefit analysis (King and Schrems, 1978) 
to techniques that attempt to include the intangible and 


qualitative benefits of the DSS. 
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First among these is Value Analysis (Keen, 1981; Money 
et al., 1988). An important premise of this approach is 
that the perceived benefits of the DSS are significant 
determinants in justifying a specific DSS. 

Keen and Scott Morton (1978) advocate a smorgasbord 
approach to determining effectiveness. Eight methodologies 
to be matched to a specific situation are proposed. These 
include examining decision outputs; changes in decision 
process; changes in managers’ concepts of the decision 
Situation; procedural changes; classical cost/benefit 
analysis; service measures; managers’ assessment of the 
system’s value; and anecdotal evidence. 

Adelman (1992) suggests that an eclectic approach is 
required to test and evaluate DSSs effectively. He defined 
three alternative types of evaluation procedures: objective 
measurement, expert observation, and subjective judgment. 
Any one or combination of these methods could be used 
depending upon the system and what the system was to 
achieve. 

A fundamental aim of an organizational information 
system (IS) is to improve individual decision-making 
performance, and ultimately organizational effectiveness. 
The difficult in empirically assessing system effectiveness 


in this way has led researchers to adopt surrogate 
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constructs that are more easily measured. Of the two main 
approaches for evaluating IS success, the first one is 
behavioral and focuses on systems usage. This approach is 
often used in empirical research (Baroudi et al., 1986; 
Gremillion, 1984; Hogue and Watson, 1985; Raymond, 1985). 
Here the implication is that if the information system helps 
improve decision guality, then the end user will use the 

Sy oreem . 

The second approach in evaluating success centers on 
user attitudes, more specifically on user satisfaction with 
various aspects of an information system (Lee, et al., 1995; 
Satlan, 224, owlmk, 1995; Hogue and Watson, 1985). End 
user IS satisfaction is the extent to which users believe 
the system meets their information requirements (Ives, et 
al., 1983). IS satisfaction is assumed to be a good 
substitute for objective determinants of information system 
success. The basic idea is that satisfied users should 
perform better than dissatisfied users and if the IS helps 
users perform better, the system is successful (Gatian, 
hoo) 

Other research, going beyond the user satisfaction with 
the system, has focused on SacisraceiGhmwrrl: InkOKmats em 


quality. Gatian’s (1994) findings support the theory that 
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availability of relevant information improved decision 
performance. 

Yet another method that has been proposed looks at the 
process. This methodology attempts to trace the effects of 
the system through all stages of the decision process. The 
focus is on the outcomes of the process and its individual 
steps (Vetschera and Walterscheid, 1995). 

We believe that a combination of these methods is 
necessary to determine the success of ha ee A model that 
combines the process orientation with the information 
quality methods is that of task-technology fit (Goodhue, 
1995). The essence of this model is that task-technology 
fit is presumed to lead to higher performance. Systems that 
provide information necessary to a user’s tasks, at the 
right level of detail, clearly and unambiguously will be 
highly valued. We intend to take this research one step 


EUREhewaand apply ier Owaespeci fic else. 


B. MEASURING LIFE CYCLE COSTS 


The LMDSS has been identified as a tool to reduce life 
cycle costes {(IMDSS Req: Doce> 199sie5) in this seerion we 
provide a review of literature to support the significance 
of measuring life cycle cost versus alternative program 


Meaolhes. 
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Pie reeceme combinagson of economic yerends, rising 
inflation, products and system cost growth, the continued 
reduction of buying power, and budget limitations has 
increased the awareness and interest in total system cost. 
Not only are the acquisition costs associated with new 
systems rising, but the costs of operating and maintaining 
a eeilowal ready in usc = are Increasing at alaming mates 
(NAVAIR TEAM,@1998; Hekoeclon 199%). BEhe requirementato 
increase overall productivity in a resource-constrained 
environment has placed emphasis on all aspects of the system 
or product life cycle. In the past, total system cost has 
not been readily visible, particularly those costs 
associated with system operations and support. As these 
cost elements are increasingly visible through computerized 
tracking and activity-based costing, they can more readily 
be managed. Further, when addressing total cost, experience 
has shown that a major portion of the projected life-cycle 
Bost FOr a given system or producti is a result or the - 
consequences of decisions made during the early phases of 
program planning and system conceptual design (Blanchard, 
1992). Decisions at this point have a major impact on 
activities and operations in all subsequent phases of the 


imfe cycle, 
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Blanchard (1992) relates the cost visibility problem to 
eWem seceberg €fiect i meihe aequisitionveest omercscacem 
design, test, pro@uction, an@econstructioneare wisible:abeve 
the water. The mass of the iceberg below the surface 
illustrates additional, less visible costs such as: 

® operations cost (personnel, facilities, utilities, 
and energy); 

* Produce Gistributlon Cost (cranspereaenenm, 
traffic, material handling) ; 

e software cost (operating and maintaining 
software) ; 

e maintenance cost (consumer service, supplier 
factory maintenance) ; 

e test and support equipment cost; 

e technical data cost; 

e supply support cost (spares, inventory, material 
Soo Ela: 

e training cost (operator and maintenance training); 

e retirement and disposal cost. 

The greatest opportunity to influence total cost, which 
18 predominantly made up of the costs illustrated as those 
costs below the water, is during the early phases of a 


program. Decisions relating to the evaluation of 
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alternative operational use profiles, maintenance and 
Support policies, equipment packaging and transportation 
schemes, and level of repair concepts have a great impact on 
total cost Ameoverarching goal is to ficeléwhigh-quality 
products, systems, and structures in response to established 
needs. It is through a concurrent life-cycle approach that 
iomige~smcanmmacealewith allveconomic factors (Pabryem and 
Blanchard, 1991) and recognize the life-cycle implication 
associated with almost all decisions. Efficient and 
effective decisions result from analysis of the total 
program (cost, performance, schedule, and political 
elements) relative to the total life cycle (concept design 
and requirements planning, design and development, 
PeOGuerlOn, = Utila zacdon, anc retinement/drsposal)” 

Product or system life-cycle analysis can be applied to the 
evaluation of numerous alternatives, including: 

1) operational scenarios and utilization approaches; 

2) system maintenance concepts and logistic support 
policies; 

3) design configurations, technology applications, 
built in test versus external test, reliability 
versus maintainability, or levels of repair versus 
discard decisions; 


4) supplier sources; 
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5) number of inventory points and levels of inventory, 

transportation and handling methods; 

6) inspection and test policies; and 

/)@product®recyelang andvdisposalmecnods ss Pabryc 

and Blanchard, 1991) 

We follow = Pabryeky and Blanchard (19919 @inm hellding thar 
an IMPOGGAWieaieast Sicepein the dmalysis is clavigyving ee 
analysis objectives. It is important to define the issues 
of concern and bind the problem such that the study can be 
efficient. Too large an effort can become overwhelming and 
it is easy to proceed in the wrong direction. The problem 
must be defined clearly, precisely, and presented in such a 
manner as to be easily understood by all concerned. 
Otherwise, it is unlikely an analysis of any kind will be 
meaningful. Within the established bounds and constraints, 
all possible alternatives should be considered, with the 
most likely candidates selected for further evaluation. 
Alternatives not considered cannot be adopted; therefore, it 
is better to consider all candidates even those that do not 
seem attainable or likely rather than overlook one that may 
be good. 

One of the greatest challenges facing industry, 
government agencies, the Department of Defense, and the 


general consumer of products and services is the growing 
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need for more effective and efficient management of our 
resources. The Department of Defense logistics mission is 
“to provide responsive and cost-effective support to ensure 
readiness and sustainability for the total force in both 
peace and war.” (USD(A&T), 1998) The fact that logistics 
costs incurred during the operating and support phase are 
such a large part of total cost requires logistics to assume 
a major role during operational use. Given the cause-and- 
effect relationships between early planning and later costs, 
logistics has become equally significant in every phase of 
the life cycle. For these reasons research, design, 
production, logistics, and system performance analyses must 
be addressed early, concurrently performed, and integrated 
throughout the system or product life cycle. 

The above discussion demonstrates the value of measuring 
life cycle costs and the value of logistics in life cycle 
cost analysis. The challenge then becomes how to measure 
and evaluate logistics performance. Good measurements should 
cover all aspects of the process being measured, be 
TicnOo-lleewrOorecach SiEUaGLION, Minimize measumement Error, 
and be consistent with the management reward system (Menzer 
and Konrad, 1991). Fabrycky and Blanchard (1991) recommend 
the cost breakdown structure (CBS) to provide a framework 


for defining life-cycle costs and communicating links for 
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Cost reporting, analysis and cost control me>s Jsecewa, er 
classifying costs with the classification being life-cycle 
oriented. The CBS links objectives and activities with 
resources andysets ip a Wegqical "subd temenmer costee 
functional activity area and major element of a system. It 
can be used as a basis for assessing the life-cycle cost of 
each alternative being considered. In logistics management, 
as with other management decisions, optimal solutions are 
often based on more than simply the financial bottom line. 
First and foremost systems must measure up to operational 
demands. The decisions that support those demands must also 


be fiscally responsible. 
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V. DATA COLLECTION TECHNIQUES AND 
METHODOLOGY 


A. CONDUCT OF THE STUDY 


Several techniques are commonly used to obtain 
perceptions, opinions, and judgments from subject matter 
experts. These generally fall into two categories: personal 
interviews and questionnaires. (Adelman, 1992) Both of these 
techniques were employed in the course of this study. 
Telephone interviews were conducted with logistics managers. 
These interviews were directed and structured, but allowed 
for open-ended responses in a number of specific areas. 

A structured questionnaire and copy of the 
interviewer’s notes from the telephone interview were sent 
to each respondent after the telephone interview. Included 
were self addressed and stamped envelopes for the surveys to 
be mailed back. We also used follow-up e-mail and telephone 
calls in an effort to improve the response rate. 

We were also fortunate enough to be able to spend a 
week at NAVAIR. During that time, we were able to conduct 
face-to-face interviews with the PMA representatives that we 


had been unable to reach by telephone. Additionally, we 
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were afforded extensive briefings on the LMDSS capabilities, 


structure, data elements, development and history. 


B. THE SAMPLE 


We selected logistics managers as targets for our 
study. This group has been specifically identified by the 
LMDSS development team as the targeted users of the LMDSS. 
There are a total of twelve aircraft types, support 
equipment, and engine program management teams considered 
relevant to the study. 

The primary target within each PMA was the Assistant 
Program Manager for Logistics (APML). In some cases, we 
were referred to the deputy APML, Product Support Team Leads 
or other support logisticians due to schools, retirement, 


travel or simply to provide yet another perspective. 


C. QUESTIONNAIRE DEVELOPMENT 


Our goal was to develop a questionnaire that 
ascertained the task-technology fit. In order to develop a 
questionnaire that assessed the appropriate areas a pre- 
study was conducted. We reviewed checklists and 
instructions used by logisticians, interviewed three 


logisticians and examined the LMDSS data elements. 
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The nature of this DSS evaluation was specific. 

Because we wished to elicit responses on the usefulness of 
System Charactemeoeres specific to the LMPSs, not an 
abstract system, an off-the-shelf survey instrument was not 
considered appropriate to our needs. 

A four part instrument was prepared. The first part 
was to be conducted as an interview with the logistics 
manager. This section was designed to elicit information 
aaout the tasks, decision Be nine oesy: and 
information needs of the logistician. 

The next three parts were designed to be filled out by 
the respondent. Part Two was to be filled out by current 
users of the LMDSS. It called for responses in Likert-type- 
scales. See Appendix F. Questions were separated as to 
general logistics concerns and specific LMDSS queries. 
Additionally, user evaluation of specific LMDSS functions 
and data was requested. | 

Part Three was designed for non-users of LMDSS. This 
section was identical to the general logistics concern 
section of Part Two. In addition, a checklist of possible 
reasons for not using the LMDSS was presented Wile LreCe Lon 
to check all that apply. 

Part Four was applicable to both users and non-users of 


Bic LMDSS] dhis sectiom also employed Likert-type scales to 
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elicit responses on job information needs. Listed were the 
information elements supported by the LMDSS. The respondent 
was asked to rate the usability of each element. 

Since this instrument had not been validated by 
previous research, we pretesting the survey with experienced 
logisticians. Based on their remarks, we made minor 
improvements before conducting the phone survey and then 
mailing the questionnaire to participants. The 


questionnaire is available in Appendix F. 


D. DATA ANALYSIS STRATEGY 


The survey results consisted of frequency, capacity, 
Satisfaction or usability ratings assigned to 166 individual 
factors by survey participants. Data were compiled from the 
eight survey forms and entered into the Microsoft Excel 
spreadsheet program. 

For each question a frequency of total respondents 
selecting a particular rating was recorded. Trends in the 
data (that is, rank orderings) were determined instead of 
making direct comparisons of adjacent ratings, due to the 


small sample available for the survey. 
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VI. FINDINGS 


A. APML QUESTIONNAIRES 


lable Via lasts thesArPML point of Ccontace by Eltle and 
the method by which data was received. Of the twelve total 
aircraft, support equipment, and engine program management 
teams considered relevant to the study we were successful in 
communicating with nine. The results of the written 
questionnaire are provided in Appendix G. Seven of the 
eight written questionnaires received were from non-users of 
the LMDSS. We interviewed the E-2C APML and EA-6B APMLs but 
did not receive a completed written questionnaire. They 
both do not use the LMDSS directly, but receive reports from 
data analysts who use the LMDSS and other tools. All 
respondents are aware of one FUNCT LONS sOl Bem@e wun SS- 

The APMLs were selected as targets for our study. This 
group has been specifically identified as the targeted 
users of the LMDSS. We were referred to additional analysts 
mic, Legustmes specialilstsmoy cignt of the nane APME 
representatives interviewed as a result of non-standardized 
organization of the PMA offices and non-standardized 


logistics input. Six of nine logistics managers interviewed 


a 


employ civilian contractors to provide reports and analyses. 


Two of nine use NAVAIR logistics specialists. 
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) Poimenot Method of Contact 
Contact _ 

Mile lamers Warfare and ei Not contacted due to 

Simulation, Alrecraft Engine retirement of APML 

f ? 


PMA-225 Hegistics phone interview and 
Management written questionnaire 
Specialist 
T-2/A-4 
PMA] 2260 2.67256 =a 3 Or — A NA 























Noe contacted > coume 
not locate good phone 
number 


| PMA~231 | E-2, C-2 APML E-2C 


Sue 234 | A-6/EA-6 Intruder/Prowler APML EA-6B phone interview and 
questionnaire 
-241 | F-14 Tomcat Deputy APML 
PMA-260 | Aviation Support Equipment phone interview and 
rear racer aoe reefer 
PMA-261 | C/MH-53E and Executive APML H-53 
PMA-265 | A F/A-18 Hornet APML 


PMA-276 | Light/Attack Helicopter Deputy APML | phone interview and 

forse written questionnaire 
upgrades 
Logistics 
Management 
Specialist, 


PMA-299 | H-2/H-60 Multi-Mission NA APML away at school. 
Helicopter 


Table VI-1 PMA Points of Contact 














































Personal and phone 
interview; written 
questionnaire 


Maritime Surveillance 
Aircraft (MSA) 





Six of nine respondents work with both new aviation 
systems and sustaining existing systems. One works only 
With new aviation systems and two work only with sustaining 
existing systems. Table VI-2 shows the frequency and 


Capacity respondents work with a number of tools while 
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performing their job. Table VI-3 shows the frequency and 
capacity with which respondents interface with project 
engineers, analysts, those who influence the budget, and 
GEhers whites rpcmuermame tEheir job. 

The responses of the single respondent who uses the 
LMDSS is provided in Table VI-4, Table VI-5, and Table VI-6 
Bice oUMBia eae eWeeUScrUna Que questions. The  respomecs © 
the non-user respondents to non-user unique guestions are 
summarized in Table VI-7. 

All responses are included in the data in Table VI-8 
Experience With Data Sources Other Than LMDSS, Table VI-9 


Logistics Concerns, and Table VI-10 Job Information Needs. 


B. RDBMS OR DSS? 


In Chapter IV (Lit Review) we presented a detailed DSS 
definition. A comparison of ther LMPSSivacthiecgas definition 
leads to the following findings: 

Le Data Management Component 

The LMDSS fully meets these component criteria. The 
NALDA II Oracle RDBMS creates an IDE with multiple data 
EPOUbCES ee eunoceauery Cabability — while limited to certian 
BSerS = Sasmciameae Le, 

Ze Model Component 


ThesuMbS> Nas enosnodolingscapability selne.  LMDSS. does 


oo 


FREQUENCY OF USE CAPACITY OF USE 


t know 
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TOOL 


Logistics Support Analysis 
(MIL-STD-1388) 
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Intuition/Experience 





Table VI-2 Logistics Management Tools 


INTERFACE FREQUENCY CAPACITY 


Interface with project engineers 

Interface with project analysts 

Interface with those who . 

influence the budget 

Interface with others: 

Depot Maintenance 

Publication Coordinators 

Type Commanders/ 
Functional Wings 

NAVICP 

Suppliers (commercial, organic) 

Integrated Logistics 4 
Support specialists 

Contracts Office 





Table VI-3 Logistics Management Interfaces 
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THE LMDSS QUERY 


Summary data; End item 

Reliability summary parameters 
Supportability summary parameters 
Cost summary parameters 

Trend analysis (problems and causes) 


Component and/or end item cost data; 
specifically: 

Annual Operations and Support Costs 
Labor Cost History 

Item Value to Depot Repair Cost 
Budget Analysis (OP-20 Report) 
Inflation Factors 

Item Value to Labor Cost 


Candidate Identification Function 
specifically: 

Detailed Component Report 
Wholesale System Demand 
Matenal Issue Trends 

Supply Synopsis 

Wholesale System Investment 
Average Customer Wait Reports 
Backorder History Reports 
NAVICP NSN Snapshot 

Mean Flight Hour Between Failure Report 


Engine Repair Cost 

Flight Hours Since Last Engine Repair 
Engine Demand Forecasting — 
Engine Overview 


Reference Information, specifically: 
Aircraft Inventory and Fatigue Life 

Code Definition 

Aircraft Engine Installation and Ownership 
Production Load and Run Statistics 
Possible Courses of Action 

Organization Codes/Job Count 
NIIN/CAGE/Part Number Cross Reference 
TEC Information 

SALTS File Information 

Data Dictionary 


FREQUENCY OF USE 


daily 
daily 
daily 
daily 
daily 


daily 

daily 

daily 
infrequently 
infrequently 
infrequently 


daily 
infrequently 
daily 
infrequently 
infrequently 
infrequently 
infrequently 
daily 
daily 


infrequently 
infrequently 
infrequently 
infrequently 


infrequently 
infrequently 
infrequently 
infrequently 
infrequently 
daily 
daily 
daily 
infrequently 
infrequently 





Table VI-4 The LMDSS Queries 
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FUNCTION RESPONSE 


Interface like 
Help Function ' like 
Analysis Tools like 
Report Presentation/Format like 


Time Required to get what is needed dislike 
Ease of getting what is needed dislike 
Training strongly dislike 
Accessibility (when desired) neutral 
Accessibility (server access) neutral 
Accessibility (password access) like 
Provides information needed like 





Table VI-5 The LMDSS Functions 


EXPERIENCE RESPONSE 


Data meet needs agree 


Data accessible agree 

Data accurate/consistent Strongly disagree 
Data detailed enough agree 

The exact data meaning is clear disagree 





Table VI-6 Experience with the LMDSS Data 
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NUMBER OF RESPONSES 


Didn't Know it existed 

PC won't support the LMDSS 

Received no training 

Don't need the information the LMDSS provides 

It takes too long to get what is needed 

It's too difficult to get what is needed 

It doesn't provide the information needed 

Other: Not developed for logistics (everyday) issues yet 


=~N OOO R= ND 





Table VI-7 Why the LMDSS is not used 


RESPONSE 


29ree 


EXPERIENCE 


Data meet needs 

Data accessible 

Data accurate/consistent 

Data detailed enough 

The exact data meaning is clear 
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Table VI-8 Experience with Data other than the LMDSS 
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FREQUENCY OF USE CAPACITY OF USE 


f know 
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LOGISTICS CONCERN 


Logistics critena as input to systems design 
Human factors concerns 
Failure mode, effects, and critical analysis 
Failure reporting, analysis, and 
corrective-action system 
Provisioning needs/altematives 
Compatibility with existing system 
Configuration Management 
Training and training support 
Manpower and personnel 
Supply Support, spares 
Inventory level analysis 
Transportation, packaging or storage 
Test equipment 
Support equipment 
Computer resource support 
Faciltties, requirements 
Facilities, location 
Data, reports requirements 
Maintenance planning 
(scheduled versus unscheduled) 
Level of repair analysis 
(O versus | versus D level) 
Operating environment issues 
Cost-drivers 
Readiness degraders 
Cycle time to repair components 
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Table VI-9 Logistics Management Concerns 
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HOW USEFUL IS THE INFORMATION ELEMENT? 


$ SS 
° et RK & wy x 
> es x »S DS) ye 
ca wy se RS & 
JOB INFORMATION ELEMENT SF LT LHL SF S 
Summary data; End item 0 0 0 2 4 1 
Summary data; Claimant 0 0 2 1 2 2 
Summary data; Organization 0 1 0 1 2 0 
Summary data; BCM Report 0 1 0 2 4 0 
Reliability summary parameters 0 0 0 1 6 0 
Supportability summary parameters 0 0 0 ‘4 5 1 
Cost summary parameters 0 0 0 1 6 0 
Emerging problems 0 0 0 0 7 0 
Common Equipment 0 0 0 1 4 2 
Trend analysis (problems and causes) 0 0 0 1 5 1 
Component and/or end item cost data; specifically: 
Annual Operations and Support Costs 0 0 0 0 ¢ 0 
Labor Cost History 0 0 0 0 7 0 
ltem Value to Depot Repair Cost 0 0 0 0 2 2 
Budget Analysis (OP-20 Report) 0 0 1 2 3 1 
Inflation Factors 0 0 1 3 2 1 
Item Value to Labor Cost 0 0 0 2 3 2 
Candidate Identification Function; specifically: 
Detailed Component Report 0 0 2 0 4 1 
Wholesale System Demand 0 0 1 4 1 1 
Matenal Issue Trends 0 0 q 2 3 1 
Supply Synopsis 0 0 1 1 4 1 
Wholesale System Investment 0 0 1 3 2 1 
Average Customer Wait Reports 0 0 1 3 3 0 
Backorder History Reports 0 1 1 2 Si 0 
NAVICP NSN Snapshot 0 1 0 2 4 0 
Mean Flight Hour Between Failure Report 0 0 0 1 5 0 
Wait Time Maintenance Impact 0 0 1 2 3 1 
Average Days to Receipt 0 0 1 3 3 0 
Planned versus Actual Opportunity Costs 0 0 1 2 2 2 
Engine Repair Cost 0 0 0 0 2 1 
Flight Hours Since Last Engine Repair 0 0 0 1 4 1 
Engine Demand Forecasting 0 0 1 0 4 1 
Engine Overview 0 0 1 1 2 2 
Engine Removal Trend 0 0 1 0 4 1 
Flight Hours Since Engine Repair at Removal 0 0 1 0 3 2 
Reference Information, specifically: 
Aircraft Inventory and Fatigue Life — 0 1 1 1 3 0 
Code Definition 0 0 1 Z 2 1 
Aircraft Engine Installation and Ownership 0 1 1 1 2 1 
Production Load and Run Statistics 0 0 1 4 1 1 
Possible Courses of Action 0 0 1 0 4 2 
Organization Codes/Job Count 0 1 2 2 2 0 
NIIN/CAGE/Part Number Cross Reference 0 0 1 2 4 0 
TEC Information 0 0 1 3 S 0 
SALTS File Information 1 0 3 1 0 2 
Data Dictionary 0 0 0 4 1 2 


Table VI-10 Logistics Management Information Elements 
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offer a module entitled Trend Analysis. This module 
provides histerieal data im tabular formae. Historsecal cere 
is also presented in a format that could support time-series 
forecasting. This is found in Engine Demand Fosecasting mun 
the Engine Analysis module, and Wholesale System Demand in 
the Supply Analysis module. Also within the Supply Analysis 
module are two subareas that can accept manual entries in 
some parameters. This allows for a degree of “what if” 
analysis for Mean Flight Hour Between Failures Report and 
Planned vs Actual Opportunity Cost. However, there is no 
sensitivity analysis capability available to support the 
“what if” analysis. 
S Dialogue Management Component 

The LMDSS meets, to some degree, all of the criteria of 
this component. The browser interface and hyperlinks offer 
navigational flexibility. Usage of the OLAP tool provides 
an additional degree of flexibility. Output either can be 
to the screen in HTML format or can be e-mailed to the 
requester in a format that can be imported to a spreadsheet 


application. There is no graphics capability. 


C. DATA QUALITY 


In the course of this study, three data quality issues 


were identified. 
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see Data Accessibility 

The IDE improves accessibility of data. The 
LMDSS/NALDA II database reduces the necessity of querying 
multiple disparate databases to retrieve relevant data for 
analysis. An economic analysis undertaken as part of the 
NALDA II Milestone III approval process found that the new 
system will allow all potential users to substantially 
reduce the amount of time required to identify and analyze 
problems in logistics support by incorporating data from as 
many aS nine other disparate databases which are currently 
used for analysis. A cost avoidance of $2 million is 
expected over the life cycle of the LMDSS. In the area of 
Common Equipment Analysis, there is a potential cost 
avoidance of $19 million. The IDE allows for performance 
and reliability of specific components across the whole 
spectrum of Naval aircraft to be ascertained. This data 
BeceCSSibIMIGyecidemet eExmsr prior tosthe EMDSS. (NAVAIRG/.0, 
E97 } 

There is, however, misunderstanding on the part of many 
logistics representatives of this data accessibility. A 
perception exists that the LMDSS and NALDA II will hamper or 
eliminate the analyst’s ability to access detail data (NAWC 
AD 3.0B, 1998). The Support Equipment PMA believed that the 


Cost Analysis capability would be inadequate for their needs 
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because cost data from DLA was not included. A review of 
the external interface data for NALDA II indicates that 
extensive DLA cost data will be available (Capstone, 1997). 

2 Data Consistency 

Data consistency has two aspects. Consistency between 
data retrieved under NALDA I and NALDA II, and consistency 
between modules in the LMDSS. 

Although the database which the LMDSS now reaches - 
NALDA II - is based on a data pull from NALDA I, the data 
outputs are not identical. This is because in the LMDSS, 
the data pre-processing has been improved to provide greater 
accuracy. Examples of the differences include use of 
aircraft versions in addition to TMS and revised item count 
LOGasee 

Currently, there are identified inconsistencies between 
the LMDSS modules. These are being addressed through the on 
going quality assurance process (Jones, 1998; SCR Sub- 
module, LMDSS application). 

3. Data Validity 

Data validity was a concern expressed by 75% of survey 
respondents and interviewees. The general perception was 
that data validity was currently poor and would continue to 


be poor. 
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One example that was offered by the SE PMA to 
illustrate this issue related to Maintenance Level 1 
(organizational level), Maintenance Action Forms (MAFs) for 
EOurELrTACEOrSsS: 

e 47%3 of MAFs recording down time due to Awaiting 


parts had no failed parts documented 


e 32% of MAFs attributed the failed part to the part 

number of the tow tractor 

e 2/72 MAFs recorded removal and replacement of the tow 

tractor part number. 
Problems like this arise, in part, because finding the 
correct work unit code (WUC) or part number can be a time 
consuming task. Busy maintainers memorize a few key WUCs 
and part numbers and use those regardless of the real 
discrepancy. Lack of training on the importance of data 
validity may also contribute. 

Another example was offered by the logistics management 
peecctalist for the S-3 aircraft. Awdata query to identify 
readiness degraders resulted in identifying the airframe as 
the top system degrader. Further investigation showed this 
was a result of how scheduled maintenance washes were being 
documented in an aircraft squadron. Additional stories were 
prevalent of the adverse impact of the use of inaccurate 


type equipment codes (TEC) or WUC on MAFs by maintenance 
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technicians who do not understand how the data is used. 

Seasoned data analysts know where to look to find erroneous 

data and know how to remove misleading data from logistics 

management reports. These data validity problems are caused ! 

by poor documentation at the source. This problem is 

currently being addressed by improved Validation 

SPECI ricCamiens atlweme inpue point Of ~NALCOMIS: | 
A second data validity area is cost data. The | 

maintenance cost data is incomplete. Maintenance Level 3 

(depot) was described during one interview as a “black 

hole.” The only aircraft cost data for ML3 in the database 

is aggregate cost. It is not broken out by TMS. Engine 

overhaul cost data - what ML3 charges the fleet - is 

available by TMS. The LMDSS database has placeholders for 

detailed cost data, but that data is currently unavailable 

from tie depores. This precludes total cost visibility. 
A final area of concern with data validity is the new 

SALTS processing procedures. When NSLC had cognizance of the 

SALTS data, a “scrubbing” process was used. Five areas of 

the detail data received from the fleet activities were 

compared with a 79 Record. If there were a 10% or greater 

discrepancy all detail data from that activity would be 


rejected. 
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Under the new system, the data will not be scrubbed 
prior to being loaded into the database. The rational for 
this change is two fold. First is the belief that there is 
more value to the analysts in having all of the data even if 
a small percentage of them are in error. Under the old 
processing system, when data were rejected the fleet 
aeuivity had to correct the data and then resubmit. It 
could take up to three months before the detail data was 
integrated into the database. Second is that the Most 
Probable Logic feature used when summarizing the detail data 


will identify and correct these errors. 
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VII. DISCUSSION 


A. APMLS AS USERS 


We selected the APMLs for our study because this group 
eiomeeeayspeciutical lyeidentified asmthe tCargebedmuserssor 
the LMDSS. Of the twelve total aircraft, support equipment, 
and engine program management teams considered relevant to 
the study we received seven complete responses and two 
partial responses. The partial responses were a phone and a 
Bersonalganverviewswi te iouEemeconplLeblonwot the swim tien 
questionnaire. Additionally, we conducted personal and 
phone interviews with logistics advisors and the LMDSS 
program representatives. 

Our data collection was limited by two constraints. 

The LMDSS is a prototype system and is not yet widely used, 
and the APML is only one of many potential users we 
subsequently identified during the course of our research. 
Because the tasks and needs of different users vary greatly, 
the information derived from the study of APMLs cannot be 
used to interpret the needs of other population groups 
without considerable risk. We identified the following user 


Sreeups castequallveampenrant asmAMeLs to logistics input sto 
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aviation program management decisions: IPT data analysts, 
FST data analysts, and Type Commands data analysts. These 
analysts may be Navy data analysts, government employees, or 
Civilian contractors assigned within the teams. 

As previously discussed, program management and the 
logistics input thereto are not standardized. Each PMA 
determines how he will manage his program. The organization 
may primarily be within the program office, the aircraft 
controlling custodian, or the depot maintenance engineering 
Support organizations. Additionally, program offices use a 
number of different sources for logistics support. Some 
program managers rely heavily on Navy personnel assigned 
within the program office, others rely on government 
employees from a logistics competency group in NAVAIR, while 
some contract out to commercial sources for logistics 
analyses and recommendations. Furthermore, PMAs must 
interface with support environments such as policy, process, 
facility and infrastructure organizations to develop optimal 
policies and processes; report actions taken and results; 
and obtain fleet feedback on system performance. All of 
these activities point to additional users of naval aviation 
logistics data and the LMDSS. 

When describing the APML job, respondents commonly 


referred to designing, developing, or analyzing support 
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IMfrastrueeures for alimerart and aircraft systems. I!!€COL, 
Wiechowski, H-53 APML, referred to this job task as the 
“care and feeding” of the heavy lift helos. The support 

int rastructuressenet Mimited@tro logistics coneerns. ~As 
presented in Table VI-3, logistics managers interface 
frequently (daily or weekly) with project engineers, project 
analysts, those that influence the program budget, and 


others to both identify and analyze requirements. 


ai THE LMDSS USE 


As presented in Table VI-7, the following reasons were 
most frequently cited for why the LMDSS is not used. 

e Received no training - 40% 

e Didn’t know it existed - 20% 

e It doesn’t provide the information needed —- 20% 
Training is a critical issue. Many studies of information 
technology deployment in sere have found that 
failure of these systems can be attributed to the lack of 
relevant and satisfactory training programs provided for all 
levels of end users. (Lee, et al, 1995; Nelson and Cheney, 
fel; Udo ama Davis, 1992 Training 1s an on-going effort by 
the NAVAIR training team. In addition to conducting 


training for data analysts located at Patuxent River, site 
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visits to users located at other bases have been and are 
being conducted. 

NAVAIR 3.6.2 continues to advertise the coming of the 
LMDSS. This is accomplished through the web-site and 
frequent presentations on the state of the application. As 
the training team continues to reach more potential users, 
awareness of its existence will increase. 

As pointed out in the Findings Chapter there is a basic 
discontinuity between the survey respondents’ PEreCe CEC ne os 
the data available under LMDSS/NALDA II and what will 
actually be available. When the LMDSS and NALDA II are 
fully on-line, all of the data available with NALDA I will 
be accessible. 

The way that the LMDSS has been advertised on its Web 
page since becoming available via the Web has contributed to 
this confusion. The LMDSS Web page does not identify the 
application as being a prototype. It does not identify the 
database as being not fully loaded. This has led some 
respondents to believe that what they currently see is what 
they will ultimately get. In actuality, the stated 
information needs from the questionnaires are provided by 
the LMDSSijdismGiusseussed jinemeanrt D of this Chapter. 

It is important to overcome this perception. End users 


need to regard the information systems they are using and 
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the information provided by the IS as relevant and useful 
for their job performance, if they are to accept such 
systems. (Lee, et al., 1995; Gatian, 1994) 

In the domain of DSS, the fundamental role of computer 
Support is to assist people in reaching decisions about the 
course of action to implement in a particular problem 
Situation. A user must reach a cognitive state where he 
understands the issues sufficiently well to choose to act or 
not. The computer tools must be designed to provide this 
fundamental layer of support to the user. The user has many 
things at stake - position, reputation, and self-image - and 
as such will rarely be willing to treat the computer as a 
black box, which tells him what course of action to 
implement. (Fedorowicz and Manheim, 1986) With this in mind, 
the issues of data validity and data summarization/origin 
must be addressed. 

While the distrust of the data validity 1s not unique 
to the LMDSS or NALDA II, it is a real as well as perceived 
problem. Additional training at the data input source, 
continued development of NALCOMIS Validation Specifications, 
and pursuit of Automated Maintenance Environment (AME) 
initiatives are all possible means of improving data 


welidity. 
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A, ENOrOough understanding of the oriqimnmancwee ein ciemon 
Of data i1smmecessary if users are to@ireust aneerumee uel baa 
the data resource (Brackett, 1996). With the exception of 
the Cost Analysis module, the algorithms for deriving the 
data and the data sources are not explicit. There is an on- 
line data@eterionary=available;, burminwour opinion. 1: adds 
Iittle Or no eClartey, fomtnensub)cctk: 

The lack of consistency between outputs under NALDA I 
and LMDSS/NALDA II should also be explained to the users. 
Preprocessing, NALCOMIS Validation Specifications, and the 
use of Most Probable Logic have all improved data accuracy, 
but in doing so created differences between outputs. The 
sources of these differences need to be explicit to the 
user. 

When people understand the content and meaning of all 
data, the use of those data to support current and future 
decision needs is limited only by people’s imagination. 
Improve the quality of information relative to timing, 
accuracy, relevancy, objectivity and understandability and 
the quality of the resultant decision making should be 


improved. (Stephenson, 1986; Gatian, 1994) 
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Ce LOGISTICS CONCERNS 


The logistics managers identified the following as the 
logistics concerns they work with most freguently (five or 


more respondents work with them weekly or daily). 
e Data, reports requirements - 87% 


e Supply support, spares - 87% 


e Maintenance planning - 63% 
e Readiness degraders —- 63% 
e Configuration Management - 63% 


e Support equipment - 63% 

Technical data is one of the four areas identified to reduce 
costs as presented in the Affordable Readiness Proposed 
Metrics. Spares and support equipment are elements of 
inventory, which is identified as another area to reduce 
costs. Readiness degrader analysis is central to 
reliability-based logistics and trigger-based item 
management that are used to implement the sustainment 
element of Affordable Readiness. (see Appendix A) 

Supply support, configuration management, support 
equipment, maintenance planning, and level of repair 
analysis are all specifically addressed as elements of total 
cost of ownership, maintenance concept, standardization, and 


supportability. These four factors are identified by 


ve 


ASN(RD&A) as those that must be considered to sustain 
SWpeOme and reduce costs. (see Appendix C) 

Logistics managers identified the following logistic 
e€encerns aS those they work with tease, frequency a) Ei veeor 
more respondents never or infrequently work with them). 

e Facilities, location - 75% 

e Facilities, requirements - 63% 

® lransporration, packaging er storage =o o5 


e Failure mode, effects and criticality analysis - 63% 


e Human factors concerns - 63% 
ITEMrelvely, ene Location OF facilities 15 Of l1ttlevconcemm 
to logistics managers because there is little opportunity to 
influence this decision. Examining the influences 
surrounding military base closures and realignments gives us 
a context in which to appreciate the limitations of an input 
by an individual APML or PMA to influence this factor. 
Additionally, we understand why logistics Managers 
infrequently work with facility requirements because this is 
only a concern during system acquisition, upgrade, or 
relocation. It is not a concern that impacts every stage of 
the life cycle, as other concerns do. 

Specific program traenspoeraeTon ala Nemec lin 


requirements are derived from the maintenance concept and 
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standardization factors (Blanchard, 1992). The Navy has a 
mature and responSive transportation, distribution and 
storage system. Logistics managers are more concerned with 
the maintenance concept concerns (maintenance planning, 
level of repair analysis, local repair versus transport to 
repair and return) and standardization concerns 
(configuration management and interchangeability) that 
contribute to transportation, packaging and storage 
concerns. The Navy transport Bees oe system imposes 
Constraints (weight limits, em@eic space limits, hoist upoint 
or shock requirements) within which the APML must work. We 
understand why a logistics manager will focus on the 
COneErEDULING seoncerns Garner ehan fixed constraints: 

Failure mode, effects, and criticality analysis (FMECA) 
1s a design tool and analysis method used to tailor the 
complexity of the design, identify possible system failures, 
the causes of these failures, the effects of the failure on 
the system, and the criticality in terms of safety and 
mmsston accomplishment (Blanchard, 1992). A logistician who 
is involved with the early development of a system will work 
with this concern, but logisticians who do not get a voice 
in design will not, nor will those who are working with 


Sustaining existing systems. 
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The objective of human factors analysis is to assure 
compatibility between the system physical and functional 
design features and the human element in the operation, 
maintenance, and support of the system. Human factor 
analysis is an integral part of overall system analysis. 
Operator and maintenance personnel requirements, and 
training program needs evolve from an iterative process of 
evaluation, system modification, and reevaluation. 
(Blanchard, 1992) Human factors concerns are more directly 
associated with engineering design and performance analysis 
efforts. As logistic concerns become more integrated with 
engineering design and performance concerns we can expect 
human factors requirements and criteria will increase in 


importance. 


D. JOB INFORMATION NEEDS 


Table VII-1 lists information elements indicated to be 
the most useful in performing the APML job. All of the 
elements presented in the questionnaire were useful to some 
degree to someone and there was no element added by a 
respondent. The elements included in the questionnaire but 
not listed in the table, are those that the respondents did 


not consistently indicate as either slightly useful or 
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extremely useful (five or more respondents; four or more for 
engine elements). 

The LMDSS provides information for each of the 
identified elements except Candidate Identification: Wait 
Time Maintenance Impact and Average Days to Receipt which 
are under development. As discussed in the Data Quality 
section of Chapter VI, the only aircraft cost data for 
Maintenance Level 3 (depot) in the database is aggregate 
cost. It is not broken out by TMS other than engine 
overhaul cost data. The Reference Information: Production 
Load and Run Statistics element provides information on the 
LMDSS not aviation programs. Respondents indicated the 
element is useful, but we believe the question may have 
falsely led them to believe it was for aviation programs not 
the LMDSS. The respondents did not use the LMDSS, this 
reference element was listed among aviation system reference 
elements, and respondents were not asked to clarify one 
reference information element from the other. Engine 
Information elements did not pertain to the response 
received from the Support Equipment APML. 

The LMDSS Trend Analysis module is. count-based and does 
not include Gopiilieowecapebliities.. 1O perform a regression 
analysis or analysis of possible cause and effect 


relationships the LMDSS data must be further manipulated 


Ss 


Summary Data; End Item 
Summary Data; Organization 
Summary Data; Beyond Capability Maintenance (BCM) Report 
Reliability Summary Parameters 
Supportability Summary Parameters 
Cost Summary Parameters 
Emerging Problems 
Common Equipment 
Trend Analysis (Problems and Causes) 
Component and/or End Item Cost Data, specifically: Annual Operations and Support Costs 
Component and/or End Item Cost Data, specifically: Labor Cost History 
Component and/or End Item Cost Data, specifically: Item Value to Depot Repair Cost 
Component and/or End Item Cost Data, specifically: Budget Analysis (OP-20 Report) 
Component and/or End Item Cost Data, specifically: Inflation Factors 
Component and/or Ene Item Cost Data, specifically: Item Value to Labor Cost 
Candidate Identification Function, specifically: Wholesale System Demand 
Candidate Identification Function, specifically: Material Issue Trends 
Candidate Identification Function, specifically: Supply Synopsis 
Candidate Identification Function, specifically: Wholesale System Investment 
Candidate Identification Function, specifically: Average Customer Wait Reports 
Candidate Identification Function, specifically: Backorder History Reports 
Candidate Identification Function, specifically: NAVICP NSN Snapshot 
Candidate Identification Function, specifically: 

Mean Flight Hour Time Between Failure (MTBF) Report 
Candidate Identification Function, specifically: Wait Time Maintenance Impact * 
Candidate Identification Function, specifically: Average Days to Receipt * 
Engine Repair Cost 
Flight Hours Since Last Engine Repair 
Engine Demand Forecasting 
Engine Removal Trend 
Reference Information: Aircraft Inventory and Fatigue Life 
Reference Information: Code Definition 
Reference Information: Production Load and Run Statistics** 
Reference Information: Possible Courses of Action 
Reference Information: NIIN/CAGE/PN Cross Reference 
Reference Information: TEC information 
Reference Information: Data Dictionary 


notes: 

Engine information elements do not pertain to Support Equipment APML. 
Aircraft Fatigue Life Reference Information does not pertain to all aircraft 
* indicates the LMDSS function is under development 

** provides reference information for the LMDSS not PMA program 


Table VII-1 Most Useful Information Elements 





beyond the LMDSS application. We believe these shortcomings 
reduce the utility of the Trend Analysis function. The 
Possible Courses of Action area simply includes a checklist 
of actions to consider and is not tailored to a specific 


program or system, forecast, or available data. 
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E. AFFORDABLE READINESS DECISIONS 


The proposed Affordable Readiness metrics discussed in 
Chapter II reflect efforts to more accurately capture the 
total cost of ownership of aviation systems. As we have 
discussed, cost reductions are considered in conjunction 
WLelmeiiemete, Leadginess and Sdrety considerations. The 
MMpGemiewdesiqned to be a tocol to facilitate continuous 
action by NAVAIR logistics management teams to measurable 
reduce the life cycle support costs (or the total cost of 
ownership) of aviation systems while protecting readiness. 
To measurably reduce the associated costs, the LMDSS must be 
able to measure the associated costs. The metrics proposed 
by Affordable Readiness define the required measurements. 
Mhe current architecture and capability of the LMDSS as a 
NALDA Phase II application adequately support measurement of 
the metrics associated with all proposed areas of Affordable 
Readiness except the metrics associated with safety (Class A 
mishaps). 

Dpeerediecr on im the Pite cycle support costs oF 
aviation systems will take more than the ability to measure 
associated costs. In addition to knowing what the costs are 
one must be able to analyze why and have incentives to make 


the “right” decisions. The LMDSS has the ability to provide 


SS 


useful data as defined by Affordable Readiness metrics. In 
addition to providing data, users must have confidence in 
those data. Our research did not imclude a comparison of the 
LMDSS analysis capabilities versus alternative methods of 
analyzing data. The perception of APMLs is the LMDSS is 
good at “big picture” and indicating “where to go look” but 
falls short of communicating details with ease or indicating 
“why” a system measurement is as it is (such as what is 
degrading mission capability or why a component is failing). 

Naming an application a DSS creates certain 
expectations. One of those expectations is that the DSS 
will support all three phases - intelligence, design and 
choice — of the decision making process. In order for a DSS 
to support the intelligence phase, it must provide accurate, 
timely information. The design phase includes inventing, 
developing and analyzing possible courses of action. The 
analysis capability is fulfilled by being able to answer 
“what 1£”% questions. The ability to suggest new 
alternatives is met by being able to perform goal seeking. 
The choice phase involves assistance in the selection of the 
alternative to be implemented. Generally, this is 
accomplished sywaeheaneoplimuzatileone soucIne- 

The LMDSS fully supports the intelligence phase of the 


decision making process.” Im order 16 support eacecenes 


86 


phases of the decision making process, the data must be 
exported to other applications that offer models or 
forecasting utilities. As one survey respondent commented, 
“The LMDSS can help answer the what, but it can’t help me 
with the why or the how.” 

The LMDSS provides the facility to export data to 
ether appelicaevens. Bue, if the LMDSS is to fulfildi.its 
Stated purposes of providing a repeatable decision making 
process it should offer a eaiardiees set of modeling and 


forecasting tools as part of the LMDSS application. 


Fr, PROGRAM MANAGEMENT ENVIRONMENT 


Because the LMDSS will not be introduced and cannot be 
evaluated in isolation, we briefly address the program 
management environment and culture to more fully complete 
the context of our analysis. 

The LMDSS is designed to support APMLs and in turn to 
benefit PMAs. Navy acquisition and program management is 
tied closely to the planning, programming, and budgeting 
(PPBS) process which determines which DoD requirements get 
funded and which do not. Those programs that get funding 
Survive. Those that do not perish. A GAO report of 
December 1996 made the following recommendations to improve 


opportunities to enhance DoD’s Logistics Strategic Plan: 
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To build on DoD’s existing strategic planning efforts and to have a better 

chance of achieving the major logistics system improvements that its plan 

envisions, we recommend that the Secretary of Defense direct the Deputy 

Under Secretary for Logistics to (1) ensure that future logistics plans 

include a recognition of the magnitude of the investment that is required to 

accomplish the plan’s goals, objectives, and strategies and (2) issue 

guidance to the Secretaries of the Army, the Navy, and the Air Force and 

the Director of DLA instructing the services and DLA on how to link their 

goals and budgets to the DoD logistic strategic plan’s overall goals and 

strategies. (GAO/NSIAD-97-28, 1996) 

As indicated, change to the logistics processes must 
compliment the organizational structure (i.e. be linked to 
overarching Navy goals) and adequate resources must be 
dedicated to turn strategy to action. The change must also 
consider other organizational factors such as measurements, 
control processes, and reward systems. Acquisition Reform 
initiatives direct PMAs to tailor programs, be more 
creative, and to consider the total cost of ownership 
(PoPimommswO0.2, Muiekok. 1997; box 199)/).) CONntLaty COnmtaese 
directives, the predominant focus of program management 
remains on unit cost, schedule, and design performance (Fox, 
VS Bacon, §I97). INCencives come l Nuc sO Supp Ome Orie amms 
down acquisition cost (unit cost), ensuring timely delivery 
of aviation systems (schedule), and meeting performance 


specifications. To measurably reduce life-cycle support 


costs PMAs need more than a tool with which to measure them. 
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As Kaminski, then USD(A&T), said when guestioned what he saw 
as the major improvements yet to be achieved in acquisition 
reform: 

Probably the biggest one is really being serious about addressing life-cycle 

cost. That is an area that I think we still talk about today, but I do not 

think we have followed through with serious initiatives. I still do not 

believe we have sufficient incentives in place for most program managers 

to seriously consider the life-cycle costs of their program... The incentives 

are still too much in the direction of saving near-year monies, and that 

support costs will be someone else’s problem in the out-years. (Fox, 1997) 

Kaminski tied incentive problems to the budget process. 
A program manager has to put up near-year funds (taken from 
another program areas) to make improvements and then when 
the out-year savings are realized those funds are Swept away 
and are not available to the program. (Fox, 1997) 

A logistics analyst for the S-3 aircraft annually 
updates a list of Logistics Engineering Change Proposals 
(LECPs) that has initiatives from ten years ago. The list 
documents projected total cost savings to by proposed 
investments in engineering changes. The list is kept from 
year to year because the proposals remain unfunded. Scarce 
O&S funds continue to be spent to maintain systems that 
cannot be upgraded without investment that reguire 
Precuremene Lunas etme LMESS may helpwadentify and justify 


LECPs, but it will not remedy these types of problems 


coreg Up: Mr re=evele. costs. 
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Additionally, while PMAs are encouraged to be creative, 
they are discouraged from taking risks. in fact, they are 
expected to plan carefully to manage and mitigate risk 
(BoDinse™5000.2; Conrow and Freedrvekson, 9796, eudwick, 
1992). DoD/DoN strategies recognize the need to change 
culture and well as impediments to do so (Fox, 1997; Hickok, 
1997; GAO7FaIMb=9e=10°7) 1996. CAG NStAD-oo 257 cs, 
GAO/AIMD/NSIAD-94-101). The LMDSS implementation must 
consider DoD/DoON strategies, ae reonn ene culture, and other 


organizational factors. 
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VIII. CONCLUSIONS AND RECOMMENDATIONS 


The data and information collected for this study on the 
effectiveness of the LMDSS to support logistic management 
decision-making provides ample material to draw conclusions 
pertinent to this study and identify areas that warrant 


further research. 


A. CONCLUSIONS 


Ly The LMDSS is not a Decision Support System. 

A DSS is composed of the following three interrelated 
components: data management, dialog management and model 
management. The LMDSS fully meets the data management 
component criteria. It meets, to some degree, all of the 
dialog management component criteria. The LMDSS has no 
modeling or sensitivity analysis capability. The LMDSS 
Trend Analysis module provides historical data in tabular 
format. Historical data is presented in a format that could 
support time-series forecasting, but not causal, and there 
is limited “what if” analysis capability. It isa 
relational database that improves data accessibility. 


2 There are multiple user groups who will be users 


of the LMDSS. 
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Usere groups include [PL dactcavanalysts, seo cces 
analysts, and Type Command data analysts. These analysts 
may be Navy data analysts, government employees or civilian 
contractors assigned within the teams. 

Se The LMDSS meets information needs to implement 

Affordable Readiness initiatives. 

The current architecture and capabilities of the LMDSS 
provide information and statistics associated with all 
proposed logistics management areas of Affordable Readiness. 
No additional information needs were identified by surveyed 
respondents. Lack of graphics, modeling and sensitivity 
analysis capabilities limit identification, analysis and 
comparison of Affordable Readiness initiatives. 

4. Data quality is both a real and perceived problem. 

We identified the following three data quality issues: 
accessibility, consistency and validity. The LMDSS improves 
the accessibility of data with the IDE. However, a 
perception exists that it will hamper or eliminate the 
analyst’s ability to access detail data. Data consistency 
is adequately addressed through the LMDSS Quality Assurance 
process. Poor documentation at the source degrades data 
validity and the lack of Maintenance Level 3 (depot) cost 


data precludes total cost visibility. 
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a The LMDSS effectiveness in measurably reducing 
life-cycle support costs is hampered by the environment in 
which aviation program management decisions are made. 

The LMDSS has the capability to support decisions to 
reduce life-cycle support costs, but PMAs need more than a 
tool with which to measure life-cycle costs to reduce them. 
Incentives continue to support driving down acquisition cost 
(UMPEecest), ensumIng Cimely delivery of aviation systems 
(schedule), and meeting performance specifications. The 
current environment encourages short-term decisions that 
compromise life-cycle decisions. The LMDSS can help 
identify and justify decisions to reduce life-cycle costs, 


but other factors are driving up these same costs. 


B. RECOMMENDATIONS 


1 Incorporate a standardized set of modeling tools 
and sensitivity analysis as part of the LMDSS application. 

To fully support the decision-making process, modeling 
capabilities are necessary. Providing a standardized set of 
modeling tools will ensure comparable analysis and 
comparison across aviation systems. Sensitivity analysis 
capabilities would allow analysts to more readily assess the 


impact of different decisions. 
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2: Incorporate graphics capability as part of the 


LMDSS application. 

Currently, data output is available in tabular format 
only. A DSS should support multiple methods of presenting 
output. Thistwould@adderlextisitiite, LOM@stmDpoOre, adit rerent 
users’ knowledge bases. 

S. Enhance availability of algorithm and data source/ 
summarization documentation. 

A thorough understanding of the origin and derivation 
of data is necesSary if users are to trust and fully use the 
data resource. Adding specific data source and algorithm 
information to the data dictionary is warranted. 

4. Expedite initiatives to improve data validity. 

Data validity problems are not unigue to the LMDSS and 
NALDA II. The Most Probable Logic function used in the data 
summarization improves the validity of summarized data, but 
poor documentation at the source precludes valid detail 
data. Initiatives, such as Automated Maintenance 
Environment (AME) and Optimized NALCOMIS are crucial to 
meaningful improvements in data quality. 


a: Collect and provide Maintenance Level 3 (depot) 


detail cost data. 
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Lack of detail cost data from ML 3 precludes total cost 
Vici p yee tcra COs Visteumeme7 1S etundamental to making 
intelligent life-cycle cost decisions. Although placeholders 
exist in the LMDSS database, they cannot be used until ML 3 
collects and provides this data. 

6. Align Budget Process, Reward Structure, and 
Strategic Decision efforts to support life-cycle cost 
reduction initiatives. 

Until the entire decision-making environment is aligned 
around a common goal of reducing life-cycle costs, efforts 
in this area will be fragmented and undermined by short-term 
imperatives. Program Managers must be effective advocates 
Of total cost of ownersShesp. In ordesx to accomplish this, 
they must be encouraged to take risks and be creative when 
considering life cycle costs and they must be rewarded for 


doing so. 
Cc. RECOMMENDATIONS FOR FURTHER RESEARCH 


a Evaluate modeling tools currently being used by 
logistics management teams and commercial modeling tools 
currently available. 

A comparison, analysis, and identification of the best 


set of standardized modeling tools will benefit efforts to 
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incorporate modeling capabilities te meet logistics 
management decision-making needs. 

2. Evaluate graphics capabilities currently being 
used by logistics management teams and commercial graphics 


tools currently available. 

A comparison, analysis, and identification of the best 
set of graphics tools will benefit efforts to incorporate 
nedelenigmeapabMetetes@te meet logistics management decision- 
making needs. 

Sc Conduct a survey of the newly identified users 
once the LMDSS is a production system. 

We selected the APMLs as targets for our study. 
Additional users were identified. Because the tasks and 
needs of different user groups vary, the information derived 
from this study of APMLs may not adequately transfer. Our 
study was also constrained by the fact that the LMDSS is 
still a prototype system. Evaluating the capability of the 
production system to meet user needs is warranted. 

4. Conduct a study of data validity. 

During the course of this study we identified some 
areas where data validity problems exist. Further research 


1s warranted to analyze additional data validity problem 
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areas, assess the impact, and evaluate alternative courses 


Ofoack Ton. 


De Assess the readiness for change and develop an 
organizational transition plan for implementing total cost 


of ownership initiatives. 

NAVAIR is currently attempting to change the focus from 
readiness at any cost to Affordable Readiness. Effective 
transition from one state to another is unlikely unless 
there is an adequate perceived need for change, the 
Organization structure, reward system and processes are in 
place to support that change, and the change is effectively 
managed. A study of where NAVAIR is in the process, where 


they are going and how best to get there is warranted. 


ON 





APPENDIX A: AFFORDABLE READINESS 


Affordable Readiness is a business practice with four inter- 
related elements: flexible sustainment, sustained 
maintenance planning, rightsourcing, and total cost of 
ownership. 

Flexible sustainment encourages program managers to use 
performance-based specifications; develop innovative, cost 
effective, life-cycle solutions; conduct supportability 


analyses; and improve reliability. It is implemented 
through reliability-based logistics and trigger-based item 
managemen ia. 


Sustained maintenance planning initiatives include 
reliability improvements; cycle time reductions; process 
improvements; technology insertions; and infrastructure 
improvements. 

Rightsourcing is defined as “selecting the most 
advantageous source to accomplish a specific function for a 
weapon system in its life cycle. Selection criteria 
include, but are not limited to life cycle cost, quality, 
reliability, safety, and effect on other programs. Specific 
functions may include all facets of Design, Production, 
CperallOny ee HegiselLeS Support, and Disposal of theysystem,” 
(NAVAIR, 1998) 

Total ownership costs include all costs associated with 
the research, development, procurement, operation, 
logistical support and disposal of an individual weapon 
system and the related infrastructure. 

For additional readings on Affordable Readiness, 
reliability-based logistics, and trigger-based management 
see www.nalda.navy.mil, NAVAIR Logistics, Affordable 
Readiness Link. 
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APPENDIX B: INTEGRATED PRODUCT TEAM 


In the context of a DoD acquisition program there are three 
mes Cf PPiss oveuaueaing IPT, working-level IPT and 
Peogram © PT 2 

The overarching IPT is formed for each program to 
provide assistance, and oversight as the program proceeds 
through the acquisition life cycle. It is composed of the 
PMA, Program Executive Officer (PEO), and appropriate 
component staff, joint staff and Office of the Secretary of 
Defense staff principals or their representatives. 

Working-level IPTs are composed of the PMA or his 
representative, and the appropriate staff members who can 
assist the program by providing functional knowledge and 
expertise to the program. For major programs working-level 
IPTs are generally focused on a particular discipline or 
functional area such as supportability, testing, 
cost/performance or contracting. For smaller projects one 
working-level IPT may be focused on the entire effort. The 
integrated IPT is an exception to this rule. The PMA may 
establish an integrated IPT to coordinate the activities of 
the other working-level IPTs. Ideally, the integrating IPT 
has as part of its membership one representative from each 
of the working-level IPTs who act as a linking pin with his 
own working-level IPT. Even though these teams are focused 
On a particular functional area, they are still multi- 
disciplinary. The supportability IPT should not be a team 
solely of logisticians but should have representatives from 
the disciplines that will influence the supportability of 
the item. 
Program IPTs are formed at the program level to manage and 
Sxeculee Programs. 
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APPENDIX C: FACTORS TO SUSTAIN SUPPORT 
AND REDUCE COSTS 


In accordance with ASN (RD&A) memorandum of 14 February 
1996, the following four factors must be considered by Navy 
PMAs and their IPTs, Program Executive Officers (PEOs), 
Direct Reporting Program Managers (DRPMs), NAVAIR Systems 
Commanders, and the Navy Secretariat staff in establishing 
Supportability requirements: total cost of ownership, 
maintenance concept, standardization, and supportability. 

An accurate picture of the total cost of ownership and 
cost relationships is necessary for cost reductions. Total 
cost of ownership includes all costs associated with the 
research, development, procurement, operation, logistical 
Support and disposal of an individual weapons system. It 
includes the total support infrastructure that plans, 
manages, and executes the weapons system over its full life. 
Currently, decisions focus on a specific cost element, 
budget line, or product line without considering the impact 
on the rest of the infrastructure. For example savings in 
depot maintenance may increase the number of systems 
required in the pipeline to maintain adequate resources at 
the operational level. Similarly, design changes may 
marginally improve performance but dramatically drive up 
Support equipment costs. 

The maintenance concept expresses the strategy for 
maintaining the platform and system at a defined level of 
readiness in support of the operational scenario. It 
includes preventive maintenance, corrective maintenance, and 
overhaul. Maintenance concepts for the platform, systems, 
and support equipment must consider maintainability at all 
maintenance levels and must be consistent. 

Standardization is intended to ensure the minimal 
variety and optimal interchangeability of technical 
information, training, equipment parts, and components. 
Achieving standardization is often in direct opposition to 
the use of performance specifications and commercial or 
nondevelopmental items. A balance between these two ends of 
the spectrum is obtained by using business and technical 
judgement in determining how to reduce the total cost of 
ownership. 
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Supportability requirements must fully consider life 
cycle costs including possible short life spans resulting 
from technology insertions or obsolescence. Requirements 
must also consider the risk of service period extensions. 
Planning must include the post production phase. PMAs must 
identify the most cost effective approach to supporting the 
system when fielded and assure that the required support 
elements, data, and information are developed and acquired. 
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APPENDIX D: SUPPORTABILITY STRATEGY 
STEPS 


NEOntractinegmecem- mepertabillity (NAVAIR, 2998) adentities 
five steps to be used to establish a supportability strategy 
for acquisition programs for new systems, major and minor 
modifications or upgrades, and commercial and 
nondevelopmental items. The following steps should be 
tailored for each type of acquisition program. 


1. Develop Strategy and Initial Support Requirements 

The APML’s first action is to determine the acquisition 
logistics strategy consistent with the overall program 
acquisition strategy. Major considerations in determining 
the acquisition logistics strategy are the type of 
acquisition, system complexity, acquisition phase, 
availability of historic data, and time and resources 
available. The availability, accuracy, and relevance of 
experience and historical databases on similar existing 
systems are crucial for accomplishment of some tasks. 
Available databases must be examined to determine if 
extensive work is needed to provide focus or relevancy. The 
acquisition logistics strategy should be periodically 
reviewed and updated to reflect any changes to the program. 
After the initial requirements are selected, further 
refinement is needed to concentrate effort in high leverage 
areas. Specific models and associated databases may be 
considered and identified at this time. 


2. Design Interface with Interrelated Efforts 

The APML must plan how to interface logistics 
requirements with the engineering community. Key related 
programs include reliability engineering, maintainability 
engineering, value engineering, human systems integration, 
system safety engineering, and transportability engineering. 
The acquisition logistics program is integrated with these — 
related programs to prevent duplication of analyses and data 
and to ensure that analyses are performed in a timely 
manner. Logistics data is sometimes based on, and should be 
traceable to, systems engineering activities. Design and 
performance information can be captured, disseminated, and 
formally controlled to serve as an audit trail for logistics 
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support planning, trade-off analyses, and documentation 
preparation. 


3. Select Logistics Products to be Developed and 
Delivered 

The APML must determine what acquisition logistics 
products are to be delivered and how they will be delivered 
(magnetic tape, disk, hard copy). The importance of 
acquiring the appropriate data must be emphasized in keeping 
with the evolving policies regarding specifications and 
standards reform and with the thrust to reduce data 
requirements. The right data can be critical. Unnecessary 
data is simply wasteful. 


4. Determine Supportability Costs 

After the APML has developed all tasks and data 
selection has been completed, he must determine the costs 
associated with the effort and document funding 
requirements. 


5x Finalize Acquisition Logistics Strategy and 
Document in Acquisition Logistics Plan and 
Statement of Work 
These actions are not independent, and careful review 
is required to ensure consistency. After the acquisition 
logistics plan becomes part of the procurement request for 
the end item, the contractor responds with his support plan. 
This ensures acquisition logistics will be integrated with 
the total acquisition program. 
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APPENDIX E: NALDA 


NALDA has been operational from the early 1980s. It 
evolved from a need for improved data analysis capabilities 
to support Fleet aviation weapon systems management. NALDA 
today is the Navy and Marine Corps central aviation 
maintenance and logistics automated information system. It 
provides an on-line, integrated life cycle logistics 
readiness and operational weapons systems database and tools 
to sustain critical support analysis. NALDA is accessed and 
used daily by Navy/Marine aviation headquarters, fleet and 
field activities. This system provides accessible, 
comprehensive, accurate and timely aviation logistics data 
analysis andy reporeing Capabilities CO Support fleet 
readiness, through sustainability of sophisticated and 
complex Naval Aviation weapons and associated support 
equipment and systems. NALDA applications encompass the 
logistics planning, management, administration, budgeting, 
and resource allocation in support of air weapon systems and 
related support equipment. The intent of NALDA is to 
Support naval aviation logistics as established by the Naval 
Aviation Maintenance Program’ (NAVAIR, 1997). 


A. NALDA Phase I 


The NALDA design has followed a phased architecture. 
Phase I is currently operational. The NALDA system is 
composed of hierarchical Data Base Management System 2000 
(S2K) databases. The primary source of data is the AV3M 
data received via Naval Sea Logistics Command (NSLC). 
Secondary sources come from NADEPs and ASO. It operates on 
the AMDAHL 5995 mainframe located at the Defense MegaCenter 
in Mechanicsburg PA. The telecommunications network 
presently consists of local dial-up and WATS lines. 





PoCrsadd1 Once cmlatiOneon the Naval Aviation 
Maintenance Program, see OPNAV 4790.2G. This instruction 
provides detailed requirements and guidance for all facets 
of the three levels of aircraft maintenance. 
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B. NALDA Phase II 


The current NALDA Phase I architecture is characterized 
by several proprietary stovepipe systems. These systems 
often lack interfaces, and offer redundant and conflicting 
information. Additionally, many of the applications are 
non-Year 2000 compliant. Phase II will address these 
deficiencies. 

Phase I will be migrated in two increments to a 
client/server architecture on the SP2 machines located at 
Patuxent River and employing the ORACLE RDBMS. Increment A 
Sein wore withep lens FO bring beens limes Om wilinemsiss Joe 
This discussion will focus on Increment A, as this is where 
the LMDSS capability is introduced. NALDA II users will 
establish a link to a NALDA Web Page via the Internet using 
commercial Web browsers and telecommunications software. 

NALDA II provides an Integrated Data Environment (IDE) 
which will include the functionality of the systems from 
Phase I with expanded capabilities and incorporated into new 
systems. The goal is to create and store data once and use 
it many times. Phase II will include the following: 1) a 
Logistics Support Analysis Record; 2) an accurate 
Configuration Management Information System/Joint Logistics 
Systems Center software for aviation weapons systems - 
configuration management is considered to be one of the 
fleet’s priorities to improve readiness and safety of 
flight; 3) the LMDSS, the Navy’s primary decision support 
system to achieve cost-effective logistics management, more 
timely (daily) receipt of fleet AV3M and configuration data, 
cost-effective consolidation of central, upline AV3M data 
Systems, and the ability to access centralized fleet- 
wide,near real-time, operational/readiness data from NALDA; 
4) Aircraft Inventory and Readiness Reporting System 
(AIRRS); 5) Reliability Centered Maintenance (RCM); 6) 
Visibility and Management of Operating/Support Cost Programs 
(VAMOSC); 7) Technical Data including Joint Computer Aided 
Logistics (JCALS) Interface; 8) Airborne Weapons Information 
System (AWIS); 9) Metrology Automated System for Uniform 
Recall and Reporting (MEASURE); 10) Affordable Readiness 
Métries/Tlotal Cost—Weeision SuppetE oyseem (fC—DSS))7) i) 
Level of Repair Analysis (LORA); and 12) other interfaces 
and applications as identified in life cycle documentation. 
Ultimately, the IDE, essentially a logistics data warehouse, 
will contain product definition, ILS acquisition, in-service 
management, fleet and depot maintenance, analysis, supply, 
cost, configuration management status reporting, and other 
data. All current and future NALDA applications will be 
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written against the single IDE data structure (NAVAIR, 
I Oye 


noo 





APPENDIX F: QUESTIONNAIRE 


This questionnaire is separated into four parts. Part One was used 
as part of a telephone interview. Parts Two through Four were mailed to 
the respondents to be filled out and then returned. 


PHONE INTRODUCTION 


Thank you for your contribution to the research project of Aerospace 
Maintenance Duty Officers LCDR Carolynn Snyder and LCDR Ellen Moore. We 
are currently pursuing Master of Science Degrees in Management at the 
Naval Postgraduate School, Monterey. Carolynn is a student in 
Information Technology and Ellen is a student in Material Logistics 


Support. We are analyzing logistics decision support in Navy aviation 
system program management. The quality of our review depends on your 
mow t . 


Specifically, we are looking at the LMDSS system designed to facilitate 
continuous action by NAVAIR logistics management teams to measurably 
reduce the life cycle support costs of aviation systems while protecting 
readiness. As a NALDA Phase II application, it incorporates data from 
existing maintenance, flight, cost, and material data bases into a 
repeatable decision making process. LMDSS is designed to enable 
logistics managers to answer the following: 

1) How am I doing? (performance versus plan) 

2) What are my current and future support cost and readiness 

drivers? 


eee 


What Can I do about it? 
eHow much will the solution cost? 
9) What is the payback period? 


We want to ensure LMDSS meets your needs. We intend to propose how the 
Navy can measure if LMDSS measurably reduces the life cycle support costs 
of aviation systems (LMDSS objective). This questionnaire will help us 
answer the following: 

1) Does the LMDSS architecture have the capability of satisfying 
APMLs? 

2) What information does an APML use to make decisions?, and 

3) Does LMDSS provide that information? 


iia 


PART ONE 
completed by interviewer 


A. Beaentezt£tication information: 


1. Name: 

2. Phone: 
3. e-mail: 
4. Address: 


S- UCo) iavele: 
6. Briek Gesemipetoum ore yop: 


7. DO you work with new aviation systems, 
HOt aw w(eirele) 


date: 


(Phone response from LMDSS USERS AND NON-USERS) 


sustaining existing systems, or 


B. How often do you use the following tools to perform your job? In the 
capacity of identifying requirements, analyzing requirements or both? 


N: never Id: identify requirements 
Peli hequenr ly A: analyze requirements 
Ms mene hay, Be) Dorn 

W: weekly DK: don’t know 

De daily 

DK: don’t know 


8. Logistic Support Analysis (MIL-STD-1388) 
Raw data 


10. Model (s) 


\O 


1£ SO; @whach onesie 


ijn Checklist 
if so, how was it developed? 


eZ 


zzz 


M W D/DK / ld A B/DK 
M W D/DK / Id A B/DK 
M W D/DK // ld A B/DK 


M W D/DK / ld A B/DK 


MW D/DK / ld A B/DK 


12. Intuition/experience N | 


ioe other: 
N | M W D/DK // ld A B/DK 


14. Do you know what LMDSS, Logistics Management Decision Support 
eysotem, 1s? (circle) 


Yes No 


fom Mave you previously or do you currently use LMDSS? (circle) 


Yes No 


(skip next question if previous answer is No) 
16.:If you use LMDSS, how often? Infrequently Monthly Weekly Daily Don’t Know 


17. a. How often do you interface with project engineer(s) ? 


N | M W D/DK // IdAB/DK 


EeeWwhat for? 
c. How (e-mail, phone, conference, etc.): 


18. a. How often do you interface with project analyst(s)? 


N | M W D/DK / IdAB/DK 


eeeewnat for? 
c. How (e-mail, phone, conference, etc.) 


19. a. How often do you interface with those who influence the program 
budget ? 
N | M W D/DK // IdAB/DK 


eee Who ? 
eeeevinat for? 
d. How (e-mail, phone, conference, etc.) 


meeeeochner: 


Who? N | M W D/DK // IdAB/DK 


a. 
b. What for? 
Cc. How (e-mail, phone, conference, etc.) 


21. How do you measure life cycle costs? 


‘Laie 


This questionnaire has been developed for APMLsS assigned to aviation 
system programs. Do you know of anyone else you feel would make a 
valuable contribution to our study, particularly anyone involved with 
logistics processes in aviation systems program management? 


Name: 
Phone: 
Address: 


Name: 
Phone: 
Address: 


Name: 
Phone: 
Address: 
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(MAIL) ... personalized address 


imemk you again for your comtmibution to the research project of 
Aerospace Maintenance Duty Officers LCDR Carolynn Snyder and LCDR Ellen 
Moore. As we discussed by phone (date), we are currently pursuing Master 
of Science Degrees in Management at the Naval Postgraduate School, 
Monterey. Carolynn is a student in Information Technology and Ellen is a 
student in Material Logistics Support. We are analyzing logistics 
decision support in Navy aviation system program management. The quality 
of our review depends on your input. 


Specifically, we are looking at the LMDSS system designed to facilitate 
continuous action by NAVAIR logistics management teams to measurably 
reduce the life cycle support costs of aviation systems while protecting 
readiness. As a NALDA Phase II application, it incorporates data from 
existing maintenance, flight, cost, and material data bases into a 
repeatable decision making process. LMDSS is designed to enable 
logistics managers to answer the following: 

1) How am I doing? (performance versus plan) 

2) What are my current and future support cost and readiness 

drivers? 

3) What can I do about it? 

4) How much will the solution cost? 

5) What is the payback period? 


We want to ensure LMDSS meets your needs. We intend to propose how the 
Navy can measure if LMDSS measurably reduces the life cycle support costs 
of aviation systems (LMDSS objective). This questionnaire will help us 
answer the following: 

1) Does the LMDSS architecture have the capability of satisfying 
APMLs? 

2) What information does an APML use to make decisions?, and 

3) Does LMDSS provide that information? 


Please feel free to address any questions or comments to: 
LCDR Ellen Moore/phone: 408-657-0891/email: eemoore@nps.navy.mil, 
or LCDR Carolynn Snyder/phone: 408-393-9567/email: cmsnyder@nps.navy.mil. 


The questionnaire is divided into four parts: 

Part ONE: Information provided by phone (please review and provide 
additional comments as desired). 

Part TWO: Written response from LMDSS users. 

Part THREE: Written response from those who have not used LMDSS. 

Part FOUR: Written response, job information needs (LMDSS users and 
Mom-users). 


ame 


Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 


PART TWO (LMDSS Users) 
C. How frequently do you work with the following logistics concerns? 


AgGweetonaliy indicate if it 1s in the’ Capacity of Aadentitying 
requirements, analyzing requirements or both. (circle the best answer) 


22. Logistic criteria as input to system design 
(reliability/maintainability goals/objectives) 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


pou capaere. IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Zs Humane aAChOrcsS concerns 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Weemeeamael cy * IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


24. Failure mode, effects, and critical analysis 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


remeaoa einai: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


25. Failure reporting, analysis, and corrective-action system 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Dem Cama Cia ze IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


26. Provisioning needs/alternatives 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


DI Veaeaeciey - IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 
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Zi . 


Zo . 


Zo: 


0°. 


Sale 


OZ. 


Sonpatiortlaty with existing sysmem 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


wee eabaCicy: 


Configuration Management 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


b. capacity: 


Training and training support 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


eeeeceapacity: 


Manpower and personnel 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


DeeCcapacity: 


Supply support, spares 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


ie Capacity: 


Inventory level analysis 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


ieee capacity: 


ey 


DON'T KNOW 


DON'T KNOW 


DON’T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


335 


34% 


So: 


BGs 


eae 


Bin 


Peat Co geeron Packaging (OF iol otage 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


Deeama Gimmy : 


Test equipment 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


iow CapaGciev: 


Support equipment 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


De Cadac ia. 


Computer resource support 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


De eGabaCiEy: 


Facilities, requirements 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


Dr eapacCrey. 


Bae Wiese Ss loeat..onm 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
ANALYZE BOTH 


REQUIREMENTS 


IDENTIFY 
REQUIREMENTS 


De nCaDaiGiiaya 


1a 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


Bo. 


a) 


Bel. 


a 


ae 


44. 


Data, reports requirements 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


ppeecaDacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Maintenance planning (scheduled versus unscheduled plan) 


pee LeOCuUeI Gy: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Pee Capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 

Level of repair analysis (O versus I versus D-levels) 

a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


DB. Capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Operating environment issues 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


mee capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Cost-drivers 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


wee capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Readiness degraders 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


mee capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


ag 


45. Cycle time to repair components 


a ee eee me ns NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Dew Gapagenny: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


AG@.mOENer: 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


rained GIs IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


47. How often do you use LMDSS? 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


~**x* Ti you do not use LMDSS then the next portion of this questionnaiim. 
has been sent to you in error. STOP NOW and call LCDR Ellen Moore/phone 
408-657-0691 or LCDR Carolynn Snyder/phone 408-393-9567 for the Cormeem 
questionnaire for NON-USERS. 


If you do use LMDSS, please continue with the questionnaire. 


D. How often do you use the following types of LMDSS queries? (circle 
the best answer) 
48. Summary data for end items 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
49. Reliability summary parameters 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
50. Supportability summary parameters 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON'T KNOW 
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51. Cost summary parameters 
NEVER INFREQUENTLY MONTHLY WEEKLY 


52. Trend analysis (problems and causes) 


NEVER INFREQUENTLY MONTHLY WEEKLY 


eee component and/or end-item cost data, 
ame Support Costs 
NEVER INFREQUENTLY MONTHLY WEEKLY 


54. Component and/or end-item cost data, 


NEVER INFREQUENTLY MONTHLY WEEKLY 


55. Component and/or end-item cost data, 
Depot Repair Cost 
NEVER INFREQUENTLY MONTHLY WEEKLY 


56. Component and/or end-item cost data, 
fee Z0 Report) 
NEVER INFREQUENTLY MONTHLY WEEKLY 


57. Component and/or end-item cost data, 


NEVER INFREQUENTLY MONTHLY WEEKLY 


98. Component and/or end-item cost data, 
iepor Cost 
NEVER INFREQUENTLY MONTHLY WEEKLY 


59. Candidate Identification Function, 
Report 
NEVER INFREQUENTLY MONTHLY WEEKLY 


SeeeCandidate Identification Function, 
Demand 


NEVER INFREQUENTLY MONTHLY WEEKLY 


eZ 


DAILY 


DAILY 


Speciticasaly.: 


DAILY 


Spoeimaca live 


DAILY 


specifically: 


DAILY 


specifically: 


DAILY 


specifically: 


DAILY 


Specmnrcally: 


DAILY 


DAILY 


DAILY 


DON’T KNOW 


DON’T KNOW 


Annual Operations 


DON'T KNOW 


Labor Cost History 


DON’T KNOW 


Ttem Value to 


DON’T KNOW 


Budget Analysis 


DON’T KNOW 


Infiilataonesaceors 


DON’T KNOW 


Ttem Value to 


DON’T KNOW 


specifically: Detailed Component 


DON'T KNOW 


specifically: Wholesale System 


DON’T KNOW 


Glee Canc@eapewidentiticatlon. punetlon a specifically: 


Trends 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


62. Candidate Identification Function, specifically: 


. NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


634, Candidate Identification Function, spect iivcal ly, 


Investment 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


64-2 Candidate Tdenmeatacation Funclionyespeciwmeal ly: 


Wait Reports 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
65. Candidate Identification Function, specifically: 
Reports 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


66. Candidate Identification Function, specifically: 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


67. Candidate Identification Function, specifically: 


Between Failure Report 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
68. Engine Repair Cost 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
69. Flight Hours Since Last Engine Repair 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
70. Engine Demand Forecasting 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 
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Material Issue 


DON’T KNOW 


Supply Synopsis 


DON'T KNOW 


Wholesale System 


DON’T KNOW 


Average Customer 


DON’T KNOW 


Backorder History 


DON'T KNOW 


NAVICP NSN Snapshot 


DON'T KNOW 


Mean Flight Hour 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


ot. 


eZ} 


ioe 


74. 


>. 


iC: 


nT 


Toe 


no. 


a0 


Si: 


Engine Overview 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Beterence Imteormariom:; i rarare Inventory and Fatigue Life 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Reference Information: Code Definition 
NEVER INFREQUENTLY MONTHLY WEEKLY § DAILY DON’T KNOW 


Reference Information: Aircraft Engine Installation and Ownership 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Reference Information: Production Load and Run Statistics 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Rererence Information: Possible Courses of Action 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Reference Information: Organization Codes /Job Count 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Reference Information: NIIN/CAGE/Part Number Cross Reference 
NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Pemerence Information: TEC Information 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Beterence Information: SALTS File Information 

NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Bererence Information: Data Dictionary 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON'T KNOW 


des 


E. Please describe your experience with the following LMDSS functions? 


(circle the best answer) 


Please include comments to clarify answers. 


82. Interface 


STRONGLY DISLIKE 
DISLIKE 


comments: 


Bom elp. PUNCE LON 


STRONGLY DISLIKE 
DISLIKE 


comments: 


84. Analysis Tools 


STRONGLY DISLIKE 
DISLIKE 


comments: 


NEUTRAL 


NEUTRAL 


NEUTRAL 


85. Report Presentation/format 


STRONGLY DISLIKE 
DISLIKE 


comments: 


86. Time required getting what is 


STRONGLY DISLIKE 
DISLIKE 


comments: 


NEUTRAL 


NEUTRAL 


LIKE STRONGLY 
LIKE 


LIKE STRONGLY 
LIKE 


LIKE STRONGLY 
LIKE 


LIKE STRONGLY 
LIKE 


needed 


LIKE STRONGLY 
LIKE 
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DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


87. Ease of getting what is needed 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


eemmencs: 


88. Training 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


Senuencts: 


89. Accessibility (when desired) 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


comments: 


90. Accessibility (server access) 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


comments: 


91. Accessibility (password access) 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


comments: 
92. Provides the information I need 


STRONGLY DISLIKE NEUTRAL LIKE 
DISLIKE 


comments: 
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STRONGLY 
LIKE 


STRONGLY 
LIKE 


STRONGLY 
LIKE 


STRONGLY 
LIKE 


STRONGLY 
LIKE 


STRONGLY 
LIKE 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


F. Please describe your experience using the LMDSS data currently 


available. (circle the best answer) 
Please include comments to clarify answers. 


93. Data meets my needs 


STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 


comments: 


94. Data is accessible 


STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 


comments: 


95. Data is accurate/consistent 


STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 


comments: 


96. Data is detailed enough 


STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 


comments: 
J eee CxAeeCala meameng 1S elear 


STRONGLY DISAGREE NEUTRAL AGREE STRONGLY 
DISAGREE AGREE 


comments: 


ZG 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


G. Please describe your experience using other data currently available. 


(circle the best answer) 


Please include comments to clarify answers. 


98. Data meets my needs 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


Somnencs: 


99. Data is accessible 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


moos Data is accurate/consistent 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


101. Data is detailed enough 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


102. The exact data meaning is clear 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


Please continue now with PART FOUR, Section 


AGREE 


AGREE 


AGREE 


AGREE 


AGREE 


Ie 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


J, Job Information Needs 


Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 


PART THREE (LMDSS Non-users) 
H. How frequently do you work with the following logistics concerns? 


Additionally indicate if it is in the capacity of identifying 
requirements, analyzing requirements or both. (circle the best answer) 


103. Logistic criteria as input to system design 
(reliability/maintainability goals/objectives) 
Pam neOuCney NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


DeGapacid Ey: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


L044] Human taceors concerns 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Daca pacCicy : IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


105. Failure mode, effects, and critical analysis 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Io5 (Ceyeyeleniu icy IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


106. Failure reporting, analysis, and corrective-action system 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Demeabpacity: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


107. Provisioning needs/alternatives 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


D.wesaeacily= IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


aZs 


ices compatibility with existing system 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


lee Capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


109. Configuration Management 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


ie Capecily: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


mee training and tralning support 
gee CEGUCNCY: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON'T KNOW 


me Capacity: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 


lili. Manpower and personnel 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


iBem@a pacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Pmezeoupply support, spares 
emeeerequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


meee capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


113. Inventory level analysis 
a .irequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


iaemeapacity: IDENTIFY ANALYZE BOTH DON'T KNOW 
REQUIREMENTS REQUIREMENTS 
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1142. Transportation, packaging or SEGrage 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 
Daaeapaci ty: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 
115. Test equipment 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Deeaepacicy: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


116. Support equipment 
a . frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Depeea Oa Ciay : IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


117. Computer resource support 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


b. capacity: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


118. Facilities, requirements 
a .frequency: | NEVER’ INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


De "Gapaeley: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


119. Facailitres, kocartion 
a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


D. Capac Hey: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


130 


ZO. 


Zs 


22 - 


7 3 


24: 


JL aaeye 


a 


De 


a 


1. 


a 


iD 


a 


iD 


a 


1D 


a 


iD. 


Data, reports requirements 


.frequency: 


Gapacity: 


Maintenance planning 


.frequency: 


Capacity: 


Level of repair analysis 


.frequency: 


Capacity: 


Operating environment issues 


.frequency: 


Capacity: 


Cost-drivers 
.frequency: 


capacity: 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


IDENTIFY 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


BOTH 


DON'T KNOW 


(scheduled versus unscheduled plan) 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON'T KNOW 


IDENTIFY 
REQUIREMENTS 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


IDENTIFY 
REQUIREMENTS 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


IDENTIFY 
REQUIREMENTS 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


IDENTIFY 
REQUIREMENTS 


Readiness degraders 


.frequency: 


e2paciey,; 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY 


IDENTIFY 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


ANALYZE 
REQUIREMENTS 


eS. 


BOTH 


(O versus I versus D-levels) 


BOTH 


BOTH 


BOTH 


BOTH 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON'T KNOW 


DON’T KNOW 


126. Cycle time to repair components 


a .Erequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


Deeeaeacicy: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


Zeon ner: 


a .frequency: NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


peescapacilLy: IDENTIFY ANALYZE BOTH DON’T KNOW 
REQUIREMENTS REQUIREMENTS 


128. How often do you use LMDSS? 


NEVER INFREQUENTLY MONTHLY WEEKLY DAILY DON’T KNOW 


*x*x*x Tf you use LMDSS then the next portion of this questionnaire has 
been sent to you in error. STOP NOW and call LCDR Ellen Moore/phone 408- 
657-0891 or LCDR Carolynn Snyder/phone 408-393-9567 for the correct 
questionnaire for NON-USERS. 


If you do not use LMDSS, please continue with the questionnaire. 


129. Why do you not use LMDSS? (check mark all that apply) 


I didn’t know it existed 
My PC won’t support LMDSS 
I’ve received no training 
I don’t need the information LMDSS provides 
It takes too long to get what I need 
It’s too difficult to get what I need 
It doesn’t provide me the information I need 


What information do you need that isn’t provided? 








Siz 


I. Please describe your experience uSing other data currently available. 


(circle the best answer) 


Please include comments to clarify answers. 


130. Data meets my needs 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


131. Data is accessible 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


132. Data is accurate/consistent 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


Semments: 
133. Data is detailed enough 


STRONGLY DISAGREE NEUTRAL 
DISAGREE | 


comments: 


AGREE 


AGREE 


AGREE 


AGREE 


134. The exact data meaning is clear 


STRONGLY DISAGREE NEUTRAL 
DISAGREE 


comments: 


Please continue now with PART FOUR, Section 


AGREE 


ies 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


STRONGLY 
AGREE 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


DON'T KNOW 


J, Job Information Needs 


Please mark each section with a legible pen or pencil. Attach additional 
pages as required. 


PART FOUR (Job Information Needs) 


J. Indicate how useful the following information elements are (or would 
be) in performing your job. If an element is not provided, please 
include as an addition. (circle the best answer) 


135. Summary data; End Item 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


136. Summary data; Claimant 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


137. Summary data; Organization 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


138. Summary data; BCM Report 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


139. Reliability summary parameters 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


140. Supportability summary parameters 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


141. Cost summary parameters 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


142. Emerging Problems 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


134 


143. Common Equipment 
NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 
144. Trend analysis (problems and causes) 
NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 


145. Component and/or end-item cost data, 


epee coupport Costs 


NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 
146. Component and/or end-item cost data, 

History 
NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 


147. Component and/or end-item cost data, 


Depot Repair Cost 


SLIGHTLY 
USEFUL 


NOT NEUTRAL 


USEFUL 


NOT AT ALL 
USEFUL 
cost data, 


148. Component and/or end-item 


(OP-20 Report) 


NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 
149. Component and/or end-item cost data, 
NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 
150. Component and/or end-item cost data, 

Labor Cost 
NOT AT ALL NOT NEUTRAL SLIGHTLY 
USEFUL USEFUL USEFUL 


151. Candidate Identification Function, 
Report 


SLIGHTLY 
USEFUL 


NOT 
USEFUL 


NOT AT ALL NEUTRAL 


USEFUL 


is 


EXTREMELY 
USEFUL 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


Se Camm ie aay: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


EXTREMELY 
USEFUL 


DON'T KNOW 


DON'T KNOW 


Annual Operations 


DON’T KNOW 


Laeor COS 


DON’T KNOW 


Item Value to 


DON'T KNOW 


Budget Analysis 


DON’T KNOW 


lp letilonaractons 


DON'T KNOW 


Item Value to 


DON’T KNOW 


specifically: Detailed Component 


DON’T KNOW 


152. Candidate Identification 
Demand 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
153. Candidate Identification 
Trends 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
154. Candidate Identification 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 


1552> Canaicate Fdentificataen 
Investment 


NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
156. Candidate Identification 
Wait Reports 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
157. Candidate Identification 
Reeores 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
15382) Gandida temlaent i f 1 cacao 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 
159. "Candidate TIdentificatswon 
Between Failure Report 
NOT AT ALL NOT NEUTRAL 
USEFUL USEFUL 


Binet Lom, 


SLIGHTLY 
USEFUL 


Pune t Lor, 


SLIGHTLY 
USEFUL 


FUNnCE Lom, 
SLIGHTLY 
USEFUL 


Funct ilom 


SLIGHTLY 
USEFUL 


Ban Ceaser 


SLIGHTLY 
USEFUL 


BUNCE Lom, 


SLIGHTLY 
USEFUL 


FuUnGEaom 
SLIGHTLY 
USEFUL 


FuUnCELOme 


SLIGHTLY 
USEFUL 
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specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


Sspecifiueally: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


SpeCGilideal iy: 


EXTREMELY 
USEFUL 


specifically: 


EXTREMELY 
USEFUL 


Wholesale System 


DON’T KNOW 


Material Issue 


DON’T KNOW 


Supply Synopsis 


DON'T KNOW 


Wholesale System 


DON’T KNOW 


Average Customer 


DON’T KNOW 


Backorder History 


DON’T KNOW 


NAVICP NSN Snapshot 


DON’T KNOW 


Mean Flight Hour 


DON’T KNOW 


160. Candidate Identification Function, specifically: 
Maintenance Impact 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
fem. Candidate Identification Function, specifically: 
Receipt 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
Mee Candidate Identification Function, specifically: 
Actual Opportunity Costs 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
ies. Engine Repair Cost 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
164. Flight Hours Since Last Engine Repair 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
165. Engine Demand Forecasting 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
166. Engine Overview 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
167. Engine Removal Trend 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 
168. Flight Hour Since Engine Repair at Removal 

NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY 

USEFUL USEFUL USEFUL USEFUL 


ies: 7 


Wait Time 


DON'T KNOW 


Average Days to 


DON'T KNOW 


Planned versus 


DON’T KNOW 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


DON’T KNOW 


DON’T KNOW 


DON'T KNOW 


169. Reference Information: Aircraft Inventory and Fatigue Life 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


170. Reference Information: Code Definition 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


171. Reference Information: Aircraft Engine Installation and Ownership 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY | DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


722" Reference Tmtormation: Production LToace ana kun) Stacrseics 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


173. Reference Information: Possible Courses of Action 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


174. Reference Information: Organization Codes /Job Count 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


175. Reference Information: NIIN/CAGE/Part Number Cross Reference 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


176. Reference Information: TEC Information 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY ‘ DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


177. Reference Information: SALTS File Information 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


178. Reference Information: Data Dictionary 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON’T KNOW 
USEFUL USEFUL USEFUL USEFUL 


ise 


io. OLner: 


NOT AT ALL NOT NEUTRAL SLIGHTLY EXTREMELY DON'T KNOW 
USEFUL USEFUL USEFUL USEFUL 


We value your input, thank you again for your contribution to our 
research. 


180. May we contact you to clarify or expand the information provided? 
YES NO 


Please return the guestionnaire by mail to the following by 27 March 
1998. (in self addressed stamped envelope provided) 


LCDR Ellen Moore 

SMC 1689, Herman Hall 
Naval Postgraduate School 
Monterey, CA 93943 


Please feel free to address any questions or comments to: 


LCDR Ellen Moore/phone: 408-657-0891/email: eemoore@nps.navy.mil 
or LCDR Carolynn Snyder/phone: 408-393-9567/email: cmsnyder@nps.navy.mil. 


Please include any additional comments, concerns, or questions below or 
on attached pages. 
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APPENDIX G: QUESTIONNAIRE RESULTS 


7? 


PART ONE (Phone response from LMDSS Users and Non-users) 


A Identifying Information: 
1 Name 
2 Phone 
3 email 
4 Address 
5 Job Title 
6 Brief Descnption of job 


7 Do you work with new aviation systems, sustaining existing systems or both? 


new existing 
1 2 


both 
2 


B How often do you work with the following tools to perform your job? 
In the capacity of identifying the requirements, analyzing the requirements, or both? 


8 Logistics Support Analysis (MIL-STD-1388) 


never infrequently monthly weekly daily don't know 
3 4 1 0 0 0 
9 Raw Data 
never infrequently monthly weekly daily don't know 
0 0 0 1 7 0 
10 Models 
never infrequently monthly weekly daily don't know 
4 0 0 0 2 1 
11 Checklists 
never infrequently monthly weekly daily don't know 
3 0 0 0 4 0 
12 Intuiton/Expenence 
never infrequently monthly weekly daily don't know 
0 0 0 0 8 0 
13 Other 
never  infrequenty monthly weekly daily don't know 
0 0 0 0 4 iz 
14 Do you know what the LMDSS is? 
yes no 
8 0 ; 
15 Have you previously or do you currently use the LMDSS? 
yes no 
1 7 


16 If you use the LMDSS, how often? 


infrequently monthly weekly daily don't know 
0 0 0 1 0 
17 How often do you interface with project engineer(s)? 
never infrequently monthly weekly daily don't know 
1 0 0 0 t 0 
18 How often do you interface with project analyst(s)? 
never infrequently monthly weekly daily don't know 
0 0 0 0 7 0 
19 How often do you interface with those who influence the program budget? 
never infrequently monthly weekly daily don't know 
0 1 0 7 4 0 
20 How often do you interface with others? 
never infrequently monthly weekly daily don't know 
0 0 0 2 6 0 
21 How do you measure life cycle costs? 


at 


identify 
4 

identify 
0 

identify 
0 

identfy 
0 


identify 
0 


identify 
0 


identify 
0 

identfy 

identify 
0 


identify 
0 


analyze 
1 


analyze 
1 


analyze 
0 


analyze 
0 


analyze 
1 


analyze 
1 


analyze 
1 


analyze 
1 


analyze 
0 


analyze 
0 


both 


both 


both 


both 
6 


don't know 
4 


don't know 
1 


don't know 
1 


don't know 
0 


don't know 
0 


don't know 
2 


don't know 
0 


don't know 
0 


don't know 
S 


don't know 
0 


PART TWO (LMDSS Users) 
C How frequently do you work with the following logistics concerns? 


Additionally, indicate if it is in the capacity of identifying requirements, analyzing requirements, or both. 


22 Logistics criteria as input to system design 


never infrequently monthly weekly daily don't know identfy § analyze 
0 0 0 0 1 0 0 0 
23 Human factors concerns 
never infrequently monthly weekly daily don't know identify § analyze 
0 1 0 0 0 0 0 0 
24 Failure mode, effects,and ctincal analysis 
never infrequently monthly weekly daily don't know identify analyze 
0 1 0 0 0 0 _0 0 
25 Failure reporting, analysis, and corrective action system 
never infrequently monthly weekly daily don't know identify § analyze 
0 1 0 0 0 0 0 0 
26 Provisioning needs/altematives 
never infrequently monthly weekly daily don't know identify analyze 
0 0 0 0 1 0 0 0 
27 Compatiblity with existing systems 
never infrequently monthly weekly daily don't know identify analyze 
0 0 0 0 1 0 0 0 
28 Configuration Management 
never infrequently monthly weekly daily don't know identfy analyze 
0 1 0 0 0 0 0 0 
29 Training and training support 
never infrequently monthly weekly daily don't know identfy analyze 
0 0 0 0 1 0 0 0 
30 Manpower and personnel 
never infrequently monthly weekly daily don't know identfy analyze 
0 0 0 0 1 0 0 0 
31 Supply support/spares 
never infrequently monthly weekly daily don't know identify § analyze 
0 0 0 0 1 0 0 0 
32 Inventory level analysis 
never infrequently monthly weekly daily don't know identify § analyze 
0 0 0 0 1 0 0 0 
33 Transportation, packaging or storage 
never infrequently monthly weekly daily don't know identify analyze 
0 1 0 0 0 0 _0 0 
34 Test equipment 
never infrequently monthly weekly daily don't know identfy analyze 
0 0 0 0 1 0 0 0 
35 Support equipment 
never infrequently monthly weekly daily don't know identify § analyze 
0 0 0 0 1 0 0 0 
36 Computer Resource support 
never infrequently monthly weekly daily don't know identify § analyze 
0 1 0 0 0 0 0 0 
37 Facilities, requirements 
never infrequently monthly weekly daily don't know identify analyze 
0 1 0 0 0 0 0 0 
38 Facilities, location 
never infrequently monthly weekly daily don't know identify analyze 
0 1 0 0 0 0 0 0 
39 Data, reports requirements 
never infrequently monthly weekly daily don't know identfy analyze 
0 0 0 0 1 0 0 0 
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both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


both 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
8) 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


40 Maintenance planning (scheduled versus unscheduled plan) 


never infrequently monthly 
0 0 0 


41 Level of repair analysis (O versus | versus D-levels) 


never infrequently monthly 
0 1 0) 
42 Operating environment issues 


never infrequently monthly 
0 1 0 
43 Cost-drivers 
never infrequently monthly 


0 0 0 
44 Readiness degraders 
never infrequently monthly 
0 0 0 
45 Cycle time to repair components 


never infrequently monthly 
0 0 0 
46 Other 
never infrequently monthly 


0) 0 0 
47 How often do you use LMDSS? 
never infrequently monthly 
0 0 0 
48 Summary data for end items 
never infrequently monthly 
0 0 0 
49 Reliability summary parameters 
never infrequently monthly 
0 0 0 


50 Supportability summary parameters 


never infrequently monthly 
0 0 0 
51 Cost summary parameters 
never infrequently monthly 
0) 0 0 


52 Trend analysis (problems and causes) 


never infrequently monthly 
0 0 0 


53 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 0 0 


54 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 0 0 


55 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 0 8) 


56 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 1 0 


5/7 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 1 0 


58 Component and/or end-item cost data, specifically: 


never infrequently monthly 
0 1 0 


identify 
0 

identify 
0 

identify 

identify 
a 

identify 
0 


identify 
0 


identify 
0 


Annual Operating and Support Costs 


weekly daily don't know 
0 1 0 
weekly daily dont know 
0 0 0 
weekly daily don't know 
0 0 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0) 0 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0 1 0 
weekly. daily don't know 
0 1 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0 1 0 
weekly daily don't know 
0 ‘T2 0 
weekly . daily don't know 
0 1 0 
Labor Cost History 
weekly daily don't know 
0 1 0 ; 
Item Value to Depot Repair Cost 
weekly daily don't know 
0 1 0 
Budget Analysis (OP-20 Report) 
weekly daily don't know 
0 0 0 
inflation Factors 
weekly daily don't know 
0 0 0 
litem Value to Labor Cost 
weekly daily don't know 
0 0 0 
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analyze 
0 


analyze 
0) 


analyze 
0 


analyze 
0 


analyze 
0 


analyze 
0 


analyze 
0 


both 


both 


both 


both 


both 


both 


both 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


59 Candidate Identification Function, specifically: Detail Component Report 
never infrequently monthly weekly daily don't know 
0 0 0 O 1 0 
60 Candidate Identification Function, specifically: Wholesale System Demand 
never ‘infrequently monthly weekly daily don't know 
0 1 0 0 0 0 
61 Candidate Identification Function, specifically: Material Issue Trends 
never infrequently monthly weekly daily don't know 


0 O 0 O 1 O 
62 Candidate Identification Function, specifically: Supply Synopsis 
never infrequently monthly weekly daily don't know 
0 1 0 O 0 0 
63 Candidate Identification Function, specifically: Wholesale System Investment 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 
64 Candidate Identification Function, specifically: Average Customer Wait Reports 
never infrequently monthly weekly daily don't know 
0 1 0 O 0 0 
65 Candidate identification Function, specifically: Backorder History Reports 
never infrequently monthly weekly daily don't know 
0 1 O O 0 0 


66 Candidate Identification Function, specifically: NIIN NSN Snapshots 
never infrequently monthly weekly daily don't know 
0 0 0 0 1 0 
67 Candidate Identification Function, specifically: Mean Flight Hour Between Failure Report 
never infrequently monthly weekly daily don't know 
0 0 0 0 1 0 
68 Engine Repair Cost 
never infrequently monthly weekly daily don't know 


0 1 0 O 0 0 
69 Flight Hours Since Last Engine Repair 
never infrequently monthly weekly daily don't know 
0 aan 0 O O O 
70 Engine Demand Forecasting 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 
71 Engine Overview 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 O 
72 Reference Information: Aircraft Inventory and Fatigue Life 
never infrequently monthly weekly daily dont know 
0 1 0 0 0 0 
73 Reference Information: Code Definition 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 
74 Reference Information: Aircraft Engine Installation and Ownership 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 


75 Reference Information: Production Load and Run Statistics 
never infrequently monthly weekly daily don't know 


0 1 0 0 0 0 
76 Reference Information: Possible Courses of Action 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 


7? Reference Information: Organization Codes/Job Count 
never infrequently monthly weekly daily don't know 
0 0 0 0 1 O 
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78 Reference Information: NIIN/CAGE/Part Number Cross Reference 


never infrequently monthly weekly daily don't know 
0 0 0 - O 1 0 
79 Reference Information: TEC Information 
never infrequently monthly weekly daily don't know 
0 0 0 0 1 0 
80 Reference Information: SALTS File Information 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 
81 Reference Information: Data Dictionary 
never infrequently monthly weekly daily don't know 
0 1 0 0 0 0 


E Please describe your expenence with the following LMDSS functions. 
82 Interface 


strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 1 0 0 
83 Help Function 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 1 0 0 
84 Analysis Tools 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 1 0 0 
85 Report presentation/format 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 4 0 0 
86 Time required getting what is needed 
strongly dislike neutral like strongly don't know 
dislike like 
0 1 0 0 0 0 
87 Ease of getting what is needed . 
strongly dislike neutral like strongly don't know 
dislike like 
0 1 0 0 0 0 
88 Training 
strongly dislike neutral like strongly don't know 
dislike like 
1 0 0 0 0 0 
89 Accessibility (when desired) 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 1 0 0 0 
90 Accessibility (server access) 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 1 0 0 0 
91 Accessibility (password access) i 
- strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 1 0 0 
92 Provides the information | need 
strongly dislike neutral like strongly don't know 
dislike like 
0 0 0 1 0 0 
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F Please descnbe your expenence using the LMDSS data currently available. 


93 Data meets my needs 


strongly disagree neutral agree 
disagree agree 
0 0 0 1 0 
94 Data ts accessible 
strongly disagree neutral agree strongly 
disagree agree 
0 0 0 1 0 
95 Data is accurate/consistent 
strongly disagree neutral agree strongly 
disagree agree 
1 0 0 0 0 
96 Data is detailed enough 
strongly disagree neutral agree strongly 
_ disagree agree 
0 0 0 1 0 
97 The exact data meaning is clear 
strongly disagree ~— neutral agree strongly 
disagree | agree 
0 1 0 0 0 


strongly don't know 


0 


don't know 


0 


don't know 


0 


don't know 


0 


don't know 


0 


G_ Please describe your experience using other data currently available. 


98 Data meets my needs 
agree 


strongly disagree neutral 
disagree agree 
0 0 0 0 1 
99 Data ts accessible 
strongly disagree neutral agree 
disagree agree 
0 0 0 0 1 
100 Data is accurate/consistent 
strongly disagree neutral agree 
disagree agree 
0 0 0 0 1 
101 Data is detailed enough 
strongly disagree neutral agree 
disagree agree 
0 0 0 0 1 
102 The exact data meaning Is clear 
strongly disagree neutral agree 
disagree agree 
0 0 0 0 1 


PART THREE (LMDSS Non-users) 


strongly don't know 


0 


strongly don't know 


0 


strongly don't know 


0 


strongly don't know 


0 


strongly don't know 


0 


H How frequently do you work with the following logistics concerns? 


Additionally, indicate if it is in the capacity of identifying requirements, analyzing requirements, or both. 


103 Logistics critena as input to system design 
never infrequently monthly weekly daily 
0 s 2 0 2 
104 Human factors concerns 
never infrequently monthly weekly daily 
0 4 1 0 2 
105 Failure mode, effects,and ctirical analysis 
never infrequently monthly weekly daily 
0 4 0 1 2 
106 Failure reporting, analysis, and corrective action system 
never infrequently monthly weekly daily 
0 1 2 2 1 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
4 
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identify 
2 

identify 
4 

identify 
1 


identfy 
4 


analyze 
1 


analyze 
3 


analyze 
2 


analyze 
4 


both 


both 


both 


both 


don't know 
0. 


don't know 
0 


don't know 
0 


don't know 
1 


107 Provisioning needs/altematives 


never infrequently monthly weekly daily 
0 1 2 2 1 
108 Compatiblity with existing systems 
never infrequently monthly weekly daily 
0 2 1 1 2 
109 Configuration Management 
never infrequently monthly weekly daily 
0 2 0 2 3 
110 Training and training support 
never infrequently monthly weekly daily 
0 1 3 2 1 
111 Manpower and personnel 
never infrequently monthly weekly daily 
0 3 4 0 0 
112 Supply support/spares 
never infrequently monthly weekly daily 
0 1 0 3 3 
113 Inventory level analysis 
never infrequently monthly weekly daily 
0 2 2 2 1 
114 Transportation, packaging or storage 
never infrequently monthly weekly daily 
0 4 0 2 1 
115 Test equipment 
never infrequently monthly weekly daily 
0 1 3 2 1 
116 Support equipment 
never infrequently monthly weekly daily 
0 1 2 2 2 
117 Computer Resource support 
never infrequently monthly weekly daily 
0 Z 2 2 1 
118 Facilities, requirements 
never infrequently monthly weekly daily 
0 4 Z 1 0 
119 Facilities, location . 
never infrequently monthly weekly daily 
0 5 2 0 0° 
120 Data, reports requirements 
never infrequently monthly weekly daily 
0 0 1 2 4 
121 Maintenance planning (scheduled versus unscheduled plan) 
never infrequently monthly weekly daily 
1 2 0 2 2 
122 Level of repair analysis (O versus | versus D-levels) 
never infrequently monthly weekly daily 
0 3 0 2 2 
123 Operating environment issues 
never infrequently monthly weekly daily 
0 Z Z 2 1 
124 Cost-drivers 
never infrequently monthly weekly daily 
0 2 3 0 2 
125 Readiness degraders 
never infrequently monthly weekly daily 
0 2 1 1 3 


don't know 
41 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
“0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
O 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 
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identify 
1 
identify 
identify 
0 
identify 
identify 
2 
identify 
0 
identify 
0 
identify 
4 
identify 
1 
identify 
41 
identify 
identify 
0 
identify 
i 
identify 
0 
identify 
0 
identify 
identify 
0 
identify 


identify 
1 


analyze 
1 


analyze 
2 


analyze 
2 


analyze 
1 


analyze 
3 


analyze 
1 


analyze 
41 


analyze 
41 


analyze 
1 


analyze 
1 


analyze 
1 


analyze 
2 


analyze 
2 


analyze 
1 


analyze 
2 


analyze 
1 


analyze 
2 


analyze 
Z 


analyze 
1 


don't know 
1 


don't know 
1 


don't know 
0 


don't know 
0 


don‘t know 
0 


don't know 
0 


don't know 
1 


don't know 
1 


don't know 
0 


don't know 
0 


don't pri 
1 


don't know 
0 


don‘t know 
1 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


don't know 
0 


126 Cycle time to repair components 


never infrequently monthly weekly daily 
0 3 2 1 1 
127 Other 
never infrequently monthly weekly daily 
0 0 1 0 1 


128 How often do you use LMDSS? 
never infrequently monthly weekly daily 


7 0 0 8) 0 
129 Why do you not use LMDSS? (check all that apply) 

2 I didn't know it existed 

1 My PC won't support LMDSS 

4 i've received no training 

0 | don't need the information LMDSS provides 

0 It takes too long to get what | need 

0 It's too difficult to get what | need 

2 It doesn't provide me the information | need 

1 other 


don't know identify § analyze both 
0 8) 1 5 
don't know identify analyze both 
0 0 6) 2 
don't know 
0 


| Please descnbe your expenence using other data currently available. 


130 Data meets my needs 


strongly disagree neutral agree strongly don't know 
disagree agree 
0 0 3 3 0 0 
131 Data ts accessible 
strongly disagree neutral agree strongly don't know 
disagree agree 
0 0 2 3 1 0 
132 Data is accurate/consistent 
strongly disagree neutral agree strongly don't know 
disagree : agree 
0 2 2 1 0 1 
133 Data ts detailed enough 
strongly disagree neutral agree strongly don't know 
disagree agree 
0 2 2 2 0 0 
134 The exact data meaning ts clear 
strongly disagree neutral agree strongly don't know 
disagree . agree 
0 2 3 1 0 0 


PART FOUR (Job Information Needs, LMDSS Users and Non-users) 
J Indicate how useful the following information elements are (or would be) in performing your job. 


135 Summary Data; End Item 


not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 0 2 4 1 
136 Summary Data; Claimant 
not at ail not neutral slightly extremely don't know 
useful useful useful useful 
0 0 2 1 2 2 
137 Summary Data; Organization 5 
not at all not neutral slighty extremely don't know 
useful - useful useful useful 
0 1 0 1 S 0 
138 Summary Data; BCM Report 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 1 0 2 & 0 
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don't know 
1 


don't know 
0 


139 Reliability summary parameters 


not at all not neutral slighty extremely don't know 
useful useful useful useful . 
0 0 0 1 6 0 
140 Supportability summary parameters 
not at all not neutral Slightly extremely don't know 
useful useful useful useful 
0 0 0 1 5 1 
141 Cost summary parameters 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 0 1 6 0 
142 Emerging Problems 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 0 0 t 0 
143 Common Equipment 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
=. 0 0 0 1 “ 2 
144 Trend Analysis (problems versus causes) 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 0 1 5 1 
145 Component and/or end-item cost data, specifically: Annual Operating and Support Costs 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 0 0 tf 0 
146 Component and/or end-item cost data, specifically: Labor Cost History 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 0 0 i 0 
147 Component and/or end-item cost data, specifically: Item Value to Depot Repair Cost 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 0 0 5 2 
148 Component and/or end-item cost data, specifically: Budget Analysis (OP-20 Report) 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 1 2 3 1 
149 Component and/or end-item cost data, specifically: Inflation Factors 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 oO 1 5 2 1 
150 Component and/or end-item cost data, specifically: Item Value to Labor Cost 
not at all not neutral Slightly extremely don't know 
useful useful useful useful 
0 0 0 2 3 Z 
151 Candidate Identification Function, specifically: Detail Component Report 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 2 0 é, 1 
152 Candidate Identification Function, specifically: Wholesale System Demand 
not at all not neutral Slightly extremely don't know 
useful useful useful useful 
0 0 1 é 1 1 
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153 Candidate Identfication Function, specifically: Matenal Issue Trends 


not at all not neutral slighty extremely don't know 
useful useful 5 useful useful 
0 0 1 2 s 1 
154 Candidate Identification Function, specifically: Supply Synopsis 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 1 4 1 
155 Candidate Identification Function, specifically: Wholesale System Investment 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
8) 0 1 . Z. 1 
156 Candidate Identification Function, specifically: Average Customer Wait Reports 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 3 3 0 
157 Candidate Identification Function, specifically: Backorder History Reports 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 1 1 2 3 0 
158 Candidate Identification Function, specifically: NIIN NSN Snapshots 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 1 0 2 4 0 
159 Candidate Identification Function, specifically: Mean Flight Hour Between Failure Report 
not at all not neutral slightly extremely don't know 
useful useful useful useful 
0 0 0 1 5 0 
160 Candidate Identification Function, specifically: Wait Time Maintenance Impact 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 2 3 1 
161 Candidate Identification Function, specifically: Average Days to Receipt 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
8) 0 1 3 3 0 
162 Candidate Identification Function, specifically: Planned versus Actual Opperating Costs 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 Z 2 ee 
163 Engine Repair Cost 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 0 0 S 1 
164 Flight Hours Since Last Engine Repair 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 0 1 4 1 
165 Engine Demand Forecasting 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 8) 4 1 
166 Engine Overview 
not at all not neutral slighty extremely don't know 
useful useful useful useful 
0 0 1 1 Zz 2 
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167 Engine Removal Trend 


neutral slightly extremely don't know 
useful useful 
1 QO. 4 1 


168 Flight Hours Since Last Engine Repair at Removal 


neutral slightly extremely don't know 
useful useful 
1 0 3 2 


Aircraft Inventory and Fatigue Life 


neutral slighty extremely don't know 
useful useful 
1 1 3 0 
Code Definition 
neutral slightly extremely don't know 
useful useful 
1 2 2 1 


Aircraft Engine Installation and Ownership 


neutral slightly extremely don't know 
useful useful 
1. 1 2 1 


Production Load and Run Statistics 


neutral slighty extremely don't know 
useful useful 
1 4 1 1 
Possible Courses of Action 
neutral slighty extremely don't know 
useful useful 
1 0 4 2 
Organization Codes/Job Count 
neutral slighty extremely don't know 
useful useful 
2 2 2 0 


NIIN/CAGE/Part Number Cross Reference 


neutral slighty extremely don't know 
useful useful 
1 2 4 0 
TEC Information 
neutral slighty extremely don't know 
useful useful 
1 3 3 0 
SALTS File Information 
neutral slighty extremely don't know 
useful useful 
3 1 O 2 
Data Dictionary 
neutral slightly extremely don't know 
useful useful 
0 4 1 Z 
neutral slighty extremely don't know 
useful useful 
0 0 0 0 


180 May we contact you to clarify or expand on the information provided? 


not at all not 
useful useful 
0 0 
not at all not 
useful useful 
0 0 
169 Reference Information: 
not at all not 
useful useful 
0 1 
170 Reference Information: 
not at all not 
useful useful 
0 0 
171 Reference Information: 
not at all not 
useful useful 
0 1 
172 Reference Information: 
not at all not 
useful useful 
0 0 
173 Reference Information: 
not at all not 
useful useful 
0 0 
174 Reference Information: 
not at all not 
useful useful 
0 1 
175 Reference Information: 
not at all not 
useful useful 
0 0 
176 Reference Information: 
not at all not 
useful useful 
0 0 
177 Reference Information: 
not at all not 
useful useful 
1 0 
178 Reference Information: 
not at all not 
useful useful 
0 0 
179 Other 
not at all not 
useful useful 
0 0 
yes no 
8 0 
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